Equipment coordination methods and related devices

Abstract

This application discloses a device collaboration method and related apparatus, applied to a first electronic device. The method includes: displaying a first user interface, the first user interface including a first icon of the first electronic device and icons of at least one detected candidate device, the icons of the at least one candidate device including a second icon of a second electronic device; the second electronic device supporting at least two collaborative capabilities, the at least two collaborative capabilities including a first capability; receiving a first input operation for selecting the first capability; receiving a second input operation; and in response to the second input operation, establishing a collaborative connection with the second electronic device supporting the first capability. In this way, for each collaboratively connectable candidate device, the capability intended for collaborative connection by the user can be selectively selected, and a collaborative connection established with that candidate device, allowing the capabilities of each candidate device to be fully and effectively utilized on the first electronic device, thereby improving the user experience.

Description

[0001] This application is a divisional application. The original application has the application number 202210191311.5 and the original application date is February 28, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of electronic technology, and in particular to device coordination methods and related apparatus. Background Technology

[0003] With the development of wireless communication technology, smartphones, tablets and other devices have begun to support users in sharing data such as pictures and documents with other devices, improving users' work efficiency and experience.

[0004] Due to limitations in their hardware architecture and system performance, different types of devices typically possess varying capabilities. For example, televisions have large displays but lack portable cameras and microphones, while mobile phones lack large displays but have portable cameras and microphones. Although current technology supports data sharing of images, videos, music, and documents between different devices, it does not support the sharing of all the capabilities supported by a single device, preventing the full utilization of each device's strengths. Users need to switch devices to utilize the capabilities of different devices, resulting in a poor user experience. Summary of the Invention

[0005] This application provides a device collaboration method and related apparatus. For each candidate device that can be collaboratively connected, a first electronic device can selectively select the capability that the user intends to collaboratively connect with, and establish a collaborative connection with the candidate device, so that the capabilities of each candidate device can be effectively and fully utilized on the first electronic device, thereby improving the user experience.

[0006] In a first aspect, this application provides a device collaboration method applied to a first electronic device, the method comprising: displaying a first user interface, the first user interface including a first icon of the first electronic device and icons of at least one detected candidate device, the icons of the at least one candidate device including a second icon of a second electronic device; the second electronic device supporting at least two collaborative capabilities, the at least two collaborative capabilities including a first capability; receiving a first input operation for selecting the first capability; receiving a second input operation; and in response to the second input operation, establishing a collaborative connection with the second electronic device supporting the first capability.

[0007] When implementing the embodiments of this application, if the second electronic device possesses multiple collaborative capabilities, the user can select any one of these collaborative capabilities from the first electronic device to establish a collaborative connection with the second electronic device. After a successful collaborative connection for the first capability, the user can invoke the first capability of the second electronic device through the first electronic device. In this way, for each currently detected candidate device capable of collaborative connection, the user can selectively choose the capability they intend to collaboratively connect with and establish a collaborative connection with that candidate device. This allows the capabilities of each candidate device to be fully and effectively utilized on the first electronic device, leveraging the advantages of each candidate device and improving the user experience.

[0008] In one implementation, the first user interface further includes: capability icons corresponding to the collaborative capabilities supported by the second electronic device; the capability icons corresponding to the collaborative capabilities supported by the second electronic device include a first capability icon corresponding to the first capability, and the first input operation acts on the first capability icon. For example, in the first user interface, capability icons corresponding to the collaborative capabilities supported by the second electronic device are displayed in the area surrounding the second icon. By implementing the embodiments of this application, through the capability icons corresponding to the collaborative capabilities supported by the second electronic device in the first user interface, the user can intuitively see which capabilities the second electronic device can collaborate with, and then select the capability corresponding to the specified capability icon as the capability to be collaborated with.

[0009] In one implementation, before receiving the first input operation for selecting a first capability, the method further includes: receiving a third input operation applied to a second icon; and in response to the third input operation, displaying the capability badges corresponding to the collaborative capabilities supported by the second electronic device in a first user interface. By implementing embodiments of this application, through a third input operation applied to the icon of a candidate device, the user can see the capability badges corresponding to the collaborative capabilities of that candidate device.

[0010] In one implementation, the method further includes: in response to a first input operation, additionally displaying a first capability badge on the second icon. By implementing embodiments of this application, a user can select an additional capability badge to display on the icon of a candidate device, and the first electronic device can establish a collaborative connection for the capability corresponding to the additionally displayed capability badge.

[0011] In one implementation, the first input operation includes an operation applied to a first capability icon; the second input operation includes: dragging a second icon toward a first icon until the second icon touches the area where the first icon is located; or, dragging a second icon toward a first icon until the distance between the second icon and the first icon is less than or equal to a preset threshold; or, dragging a first icon toward a second icon until the first icon touches the area where the second icon is located; or, dragging a first icon toward a second icon until the distance between the first icon and the second icon is less than or equal to a preset threshold.

[0012] In one implementation, the second input operation is applied to the first capability marker. By implementing the embodiments of this application, a cooperative connection supporting the first capability can be established through an input operation applied to the first capability marker.

[0013] In one implementation, the first input operation includes an operation on a first capability icon; the second input operation includes: dragging the first capability icon toward the first icon until the first capability icon touches the area where the first icon is located; or, dragging the first capability icon toward the first icon until the distance between the first capability icon and the first icon is less than or equal to a preset threshold.

[0014] In one implementation, the at least two collaborative capabilities further include a second capability; after successfully establishing a collaborative connection supporting the first capability with the second electronic device, the method further includes: receiving a fifth input operation applied to the second icon; in response to the fifth input operation, displaying capability badges corresponding to the collaborative capabilities supported by the second electronic device, wherein the capability badges corresponding to the collaborative capabilities supported by the second electronic device include a first capability badge corresponding to the first capability and a second capability badge corresponding to the second capability; receiving a sixth input operation applied to the second capability badge; and in response to the sixth input operation, establishing a collaborative connection supporting the second capability with the second electronic device. By implementing the embodiments of this application, when a collaborative connection for the first capability has been established with the second electronic device, by selecting the second capability badge from the capability badges corresponding to the collaborative capabilities of the second electronic device, a collaborative connection supporting the second capability can be further established with the second electronic device.

[0015] In one implementation, the at least two collaborative capabilities further include a second capability; the step of establishing a collaborative connection supporting the first capability with the second electronic device in response to the second input operation specifically includes: establishing a collaborative connection supporting both the first and second capabilities with the second electronic device in response to the second input operation. Implementing the embodiments of this application allows for the establishment of a collaborative connection supporting multiple capabilities with the second electronic device.

[0016] In one implementation, when receiving a second input operation, the second icon additionally displays a first capability badge corresponding to the first capability and a second capability badge corresponding to the second capability. By implementing embodiments of this application, based on the multiple capability badges additionally displayed on the second icon, a collaborative connection supporting multiple capabilities corresponding to the aforementioned multiple capability badges can be established with the second electronic device.

[0017] In one implementation, after successfully establishing a collaborative connection supporting both a first capability and a second capability with the second electronic device, the method further includes: receiving a seventh input operation applied to a second icon; responding to the seventh input operation, displaying capability badges corresponding to the collaborative capabilities supported by the second electronic device, wherein the capability badges corresponding to the collaborative capabilities supported by the second electronic device include a first capability badge corresponding to the first capability and a second capability badge corresponding to the second capability; receiving an eighth input operation applied to the first capability badge; and responding to the eighth input operation, canceling the collaborative connection with the second electronic device supporting the first capability. By implementing the embodiments of this application, after the first electronic device and the second electronic device establish a collaborative connection supporting multiple capabilities, the collaborative connection for any of the aforementioned collaborative capabilities can be canceled via the capability badge of that capability; this facilitates the user's ability to selectively cancel the collaborative connection for a specific capability, improving the user experience.

[0018] In one implementation, the first user interface further includes capability badges corresponding to at least two capabilities; the at least two capabilities include all collaborable capabilities supported by the at least one candidate device, and the capability badges corresponding to the at least two capabilities include a first capability badge corresponding to the first capability, with the first input operation acting on the first capability badge. Implementing this embodiment allows the display of capability badges corresponding to all collaborable capabilities of all candidate devices, enabling users to selectively choose the capabilities they intend to collaborate with, as well as the candidate devices they intend to connect to. This allows the capabilities of each candidate device to be fully and effectively utilized on this device, leveraging the strengths of each candidate device and improving the user experience.

[0019] In one implementation, before receiving the first input operation for selecting the first capability, the method further includes: receiving a ninth input operation applied to a first icon; and in response to the ninth input operation, displaying capability badges corresponding to the at least two capabilities in a first user interface. By implementing embodiments of this application, through the ninth input operation applied to the icon of the first electronic device (i.e., this device), the user can see the capability badges corresponding to the collaborative capabilities of all candidate devices.

[0020] In one implementation, the method further includes: in response to a first input operation, attaching a first capability badge to the first icon.

[0021] In one implementation, the method further includes: in response to a first input operation, stopping the display of icons of candidate devices that cannot collaborate with the first capability in a first user interface. Implementing embodiments of this application, when a user selects the capability to be collaborated with as the first capability via a first capability badge, the display of icons of candidate devices that cannot collaborate with the first capability is stopped, so that the user can subsequently select candidate devices to collaborate with and connect to the first capability.

[0022] In one implementation, among the capability badges displayed around the second icon, the capability badges with attached attributes to the second icon are displayed as selected, and the capability badges without attached attributes to the second icon are displayed as unselected. The method further includes: in response to a first input operation, switching the first capability badges displayed around the second icon from an unselected state to a selected state. The method further includes: in response to a fourth input operation, switching the second capability badges displayed around the second icon from a selected state to an unselected state.

[0023] In one implementation, the method further includes: when a successful collaborative connection supporting the first capability is established with the second electronic device, the second icon switches from a non-collaborative state to a collaborative state, and the second icon is additionally displayed with a first capability badge corresponding to the first capability; when a failed collaborative connection supporting the first capability is established with the second electronic device, the second icon remains in a non-collaborative state.

[0024] In one implementation, before receiving the second input operation, a first icon is displayed in a first preset area, and the icons of the at least one candidate device are displayed in a second preset area. The non-cooperative state of the second icon is presented as being displayed in the second preset area in a first state; the cooperative state of the second icon is presented as being attached to the first icon in a second state and displayed in the first preset area, or the cooperative state of the second icon is presented as being displayed in the second preset area in a second state. For example, the first state and the second state refer to different icon colors.

[0025] In one implementation, after the first electronic device performs the step of "establishing a cooperative connection with the second electronic device that supports the first capability and the second capability", when a cooperative connection supporting the first capability and the second capability is successfully established with the second electronic device, the second icon switches from a non-cooperative state to a cooperative state, and the second icon is additionally displayed with the first capability badge and the second capability badge; when a cooperative connection supporting the first capability is successfully established with the second electronic device, but the cooperative connection does not support the second capability, the second icon switches from a non-cooperative state to a cooperative state, and the second icon is additionally displayed with the first capability badge, but the second capability badge is not additionally displayed.

[0026] In one implementation, in response to the seventh input operation, among the capability badges displayed around the second icon, the capability badges corresponding to the capabilities supported by the cooperative connection established with the second electronic device (i.e., the first capability badge and the second capability badge) are displayed as selected, while the capability badges corresponding to the capabilities not supported by the cooperative connection established with the second electronic device are displayed as unselected.

[0027] In one implementation, after successfully establishing a cooperative connection supporting the first and second capabilities with the second electronic device, the method further includes: receiving a tenth input operation applied to the second icon; and in response to the tenth input operation, canceling the cooperative connection with the second electronic device for any capability.

[0028] In one implementation, in response to a third input operation, capability badges corresponding to the collaborative capabilities supported by the second electronic device are displayed in a first area; the at least two collaborative capabilities further include a second capability, and when a third input operation is received on the second icon, a second capability badge corresponding to the second capability is additionally displayed on the second icon; after receiving the third input operation on the second icon, the implementation further includes: receiving a fourth input operation on the second capability badge in the first area; in response to the fourth input operation, the additional display of the second capability badge on the second icon is stopped.

[0029] Secondly, this application also provides a device collaboration method applied to a first electronic device, the method comprising: displaying a first user interface, the first user interface including a first icon of the first electronic device and icons of at least one detected candidate device, the icons of the at least one candidate device including a second icon of a second electronic device; the second electronic device supporting at least two collaborative capabilities, the at least two collaborative capabilities including a first capability; receiving an eleventh input operation; and in response to the eleventh input operation, establishing a collaborative connection supporting the first capability with the second electronic device based on the collaboration state of a first application recently running in the foreground and / or the second electronic device.

[0030] By implementing the embodiments of this application, the first electronic device can adaptively select one or more capabilities for collaborative connection based on the current application scenario (i.e., the collaborative state of the first application running in the foreground and / or the second electronic device), so that the capabilities of each candidate device can be fully and targetedly applied on the first electronic device, giving full play to the advantages of each candidate device and improving the user experience.

[0031] In one implementation, the cooperative state of the second electronic device includes one or more of the following: the number of cooperative connections of each capability of the first and second electronic devices, the capability most recently cooperatively connected between the first and second electronic devices, and the priority of each cooperative capability of the second electronic device. This application does not specifically limit the cooperative state of the second electronic device.

[0032] In one implementation, establishing a collaborative connection for a first capability of the second electronic device based on the collaborative state of the most recently running first application and / or the second electronic device includes: establishing a collaborative connection for the first capability of the second electronic device when the collaborative capability of the second electronic device includes the first capability required by the first application. By implementing embodiments of this application, the first electronic device can adaptively select the capability to be collaboratively used based on the capability required by the most recently running first application, effectively improving the user experience.

[0033] In one implementation, establishing a collaborative connection with a first capability of the second electronic device based on the collaborative state of the most recently running first application and / or the second electronic device includes: establishing a collaborative connection with the first capability that has been collaboratively connected with the second electronic device the most times; or, establishing a collaborative connection with the first capability that was most recently collaboratively connected with the second electronic device; or, establishing a collaborative connection with the first capability with the highest priority among the collaborative capabilities of the second electronic device. By implementing embodiments of this application, the first electronic device can adaptively select the capability to be collaboratively connected based on the collaborative state of the collaborative capabilities of the second electronic device, effectively improving the user experience.

[0034] Thirdly, this application provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the device coordination method in any possible implementation of any of the above aspects.

[0035] Fourthly, embodiments of this application provide a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the device coordination method in any of the possible implementations of any of the above aspects.

[0036] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to execute the device coordination method in any of the possible implementations of any of the above aspects. Attached Figure Description

[0037] Figure 1A This is a schematic diagram of the system architecture of the communication system provided in the embodiments of this application;

[0038] Figure 1B A schematic diagram of the collaborative connection system provided in the embodiments of this application;

[0039] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0040] Figures 3A to 3D This application provides a method for invoking the collaborative connection interface in an embodiment.

[0041] Figures 4A to 4D Another way to invoke the collaborative connection interface provided in the embodiments of this application;

[0042] Figures 5A to 5D Another way to invoke the collaborative connection interface provided in the embodiments of this application;

[0043] Figures 6A to 6E The relevant user interface for collaborative connectivity provided in Scenario 1 of this application embodiment;

[0044] Figures 7A to 7C The relevant user interface for collaborative connectivity provided in Scenario 1 of this application;

[0045] Figures 8A to 8G The relevant user interface for the switching capability badge provided in the embodiments of this application;

[0046] Figures 9A to 9C The relevant user interface for collaborative connectivity provided in Scenario 1 of this application embodiment;

[0047] Figures 10A to 10F The relevant user interface for collaborative connectivity provided in Scenario 1 of this application embodiment;

[0048] Figures 11A to 11D The user interface for canceling collaborative connections provided in the embodiments of this application;

[0049] Figures 12A to 12G The relevant user interface for the switching capability badge provided in the second scenario of this application embodiment;

[0050] Figures 13A to 13J The relevant user interface for collaborative connection provided in the second scenario of this application embodiment;

[0051] Figures 14A to 14E The relevant user interface for collaborative connection provided in the second scenario of this application embodiment;

[0052] Figures 15A to 15D The relevant user interface for canceling the collaborative connection is provided in Scenario 2 of this application embodiment;

[0053] Figures 16A to 16D The user interface provided in this application embodiment is related to establishing a collaborative connection between two devices;

[0054] Figures 17A to 17F The relevant user interface for the switching capability badge provided in Scenario 3 of this application embodiment;

[0055] Figures 18A to 18C The relevant user interface for collaborative connection provided in Scenario 3 of this application embodiment;

[0056] Figures 19A to 19J The relevant user interface for collaborative connection provided in Scenario 3 of this application embodiment;

[0057] Figures 20A to 20D The user interface related to the switching capability badge provided in the embodiments of this application for Case 4;

[0058] Figures 21A to 21D The relevant user interface for collaborative connectivity provided in Scenario 4 of this application embodiment;

[0059] Figures 22A to 22D The relevant user interface for collaborative connectivity provided in Scenario 4 of this application embodiment;

[0060] Figures 23A to 23C The relevant user interface for collaborative connectivity provided in Scenario 4 of this application embodiment;

[0061] Figures 24A to 24F The user interface provided in this application embodiment is related to establishing collaborative connections with multiple devices;

[0062] Figures 25A to 25D The relevant user interface for collaborative connectivity provided in Scenario 5 of this application embodiment;

[0063] Figures 26A to 26D The relevant user interface for the application of collaborative connection capabilities provided in the embodiments of this application;

[0064] Figure 27 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0065] Figure 28 A schematic diagram of the software architecture of another electronic device provided in an embodiment of this application;

[0066] Figure 29 A schematic diagram of the software architecture of a tablet computer provided in an embodiment of this application;

[0067] Figure 30 A schematic diagram of the software architecture of a laptop computer provided in an embodiment of this application;

[0068] Figure 31 A schematic diagram of the software architecture of a printer provided in an embodiment of this application;

[0069] Figure 32 This is a flowchart illustrating a device collaboration method provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0071] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0072] First, let's introduce the communication system 10 provided in the embodiments of this application.

[0073] Figure 1A A schematic diagram of a communication system 10 provided in an embodiment of this application is shown as an example. Figure 1A As shown, the communication system 10 includes an electronic device 100 and one or more candidate devices (e.g., candidate device 200) that can establish a connection with the electronic device 100.

[0074] The electronic device 100 and candidate device 200 can be mobile phones, tablets, desktop computers, laptops, handheld computers, ultra-mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices (smart bracelets), in-vehicle devices, smart home devices (smart TVs, smart screens, large-screen devices, etc.), and / or smart city devices. This application embodiment does not impose any special restrictions on the specific type of the electronic device. Candidate device 200 can also be a projector, speaker, Bluetooth headset, monitor, mouse, keyboard, printer, etc. The electronic device 100 and candidate device 200 can run the same operating system or different operating systems, such as iOS, Android, Microsoft, HarmonyOS, etc.

[0075] In this embodiment, electronic device 100 can establish a collaborative connection with capability 1 of candidate device 200, enabling electronic device 100 to share capability 1 of candidate device 200. The functional modules corresponding to the shared capabilities in candidate device 200 can be indirectly regarded as functional modules of electronic device 100. In some embodiments, after capability 2 of electronic device 100 and candidate device 200 collaborate, candidate device 200 can also share capability 2 of electronic device 100. The functional modules corresponding to the shared capabilities in electronic device 100 can also be indirectly regarded as functional modules of candidate device 200, thereby realizing capability mutual assistance and resource sharing between electronic device 100 and candidate device 200. In this way, electronic device 100 and one or more of the aforementioned candidate devices can be regarded as a distributed collaborative super terminal. For example, electronic device 100 is a mobile phone, and users can use the mobile phone to achieve communication connections and capability collaboration with devices such as tablets, laptops, projectors, mice, keyboards, printers, speakers, and Bluetooth headsets. The mobile phone can indirectly possess the capabilities of other devices.

[0076] The aforementioned capabilities may refer to hardware functional modules (such as cameras, microphones, speakers, displays, printers, etc.), software functional modules (such as payment, image processing, Wi-Fi connection, screen projection, etc.), or functional modules combining hardware and software (such as camera functions with image processing, audio output functions with audio processing, audio input functions with audio processing, etc.). This application embodiment does not specifically limit these capabilities.

[0077] For example, such as Figure 1B As shown in this embodiment, the user interface (UI) layer of the electronic device 100 can display the device's icon, the icons of detected candidate devices (e.g., tablets, laptops, projectors, speakers, headphones, printers, etc.), and capability badges of the collaborative capabilities supported by the candidate devices (e.g., camera, audio output, screen mirroring, audio input, printing, etc.) in the collaborative connection interface. Users can select a suitable candidate device (e.g., candidate device 200) from the candidate device icons displayed on the electronic device 100 according to their needs, and select the capability badge of the capability to be collaboratively used (e.g., capability 1) from the capability badges corresponding to that candidate device; then... Subsequently, through a preset input operation (e.g., dragging the icon of candidate device 200 close to the icon of this device), the electronic device 100 achieves a collaborative connection with capability 1 of candidate device 200. After detecting the aforementioned preset input operation, the electronic device 100 monitors the connection status of the underlying services. Once the electronic device 100 detects that the collaborative connection of capability 1 of candidate device 200 is successful, it switches the display status of the icon of candidate device 200 to the collaborative state in the UI layer, and adds a capability badge indicating capability 1 to the icon of candidate device 200. When the service layer needs to call capability 1, it can call capability 1 of candidate device 200 based on the user's selection. In this way, the electronic device 100 can obtain one or more specified capabilities of candidate device 200.

[0078] It should be noted that the candidate device 200 can support at least one capability, and the candidate device 200 or the user can set some or all of the above-mentioned capabilities as collaborative connectivity capabilities of the candidate device 200 (which can also be simply referred to as collaborative capabilities in this embodiment). For example, the candidate device 200 supports capabilities such as camera, audio output function, screen projection function, and audio input function. The candidate device 200 can set only the camera and audio output functions as collaborative capabilities, while the other capabilities are non-collaborative capabilities of the candidate device 200; correspondingly, the icon of the candidate device 200 in the UI layer of the above-mentioned electronic device 100 can only be attached with capability badges for camera and / or audio output functions, and cannot be attached with capability badges for other capabilities.

[0079] In some embodiments, the collaborative connection between electronic device 100 and candidate device 200 regarding capability 1 of candidate device 200 means that, when or after establishing a communication connection (i.e., a data connection), electronic device 100 and candidate device 200 also establish a service connection for capability 1, so that electronic device 100 obtains authorization to use capability 1 of candidate device 200. In this embodiment, the user can achieve collaborative connection between electronic device 100 and candidate device 200 regarding capability 1 through the collaborative connection interface provided in this embodiment. It should be noted that if a communication connection has already been established between electronic device 100 and candidate device 200 when establishing the collaborative connection, the aforementioned collaborative connection may refer to the service connection regarding capability 1; if a communication connection has not yet been established between electronic device 100 and candidate device 200 when establishing the collaborative connection, the aforementioned collaborative connection may refer to both the communication connection between electronic device 100 and candidate device 200 and the service connection for capability 1.

[0080] In some embodiments, establishing a cooperative connection between electronic device 100 and capability 1 of candidate device 200 may include: after establishing a communication connection with candidate device 200, electronic device 100 sends a cooperative connection request to candidate device 200, the cooperative connection request carrying an identifier of capability 1 of candidate device 200; after receiving the cooperative connection request, candidate device 200 determines the capability that electronic device 100 intends to cooperate with based on the identifier of capability 1, and candidate device 200 may send a cooperative connection response to electronic device 100 to agree to the cooperative connection of capability 1. In one implementation, the cooperative connection response may include capability parameters of capability 1 of candidate device 200; after successful cooperative connection, electronic device 100 may invoke capability 1 of candidate device 200 based on the capability parameters. It is understood that in the embodiments of this application, establishing a cooperative connection with (or with) capability 1 of candidate device 200 means establishing a cooperative connection with candidate device 200 that supports capability 1.

[0081] It should be noted that when electronic device 100 itself possesses capability 1, it can also acquire capability 1 of candidate device 200 through collaborative connection and process related business through capability 1 of candidate device 200. For example, if electronic device 100 has a camera, electronic device 100 can also acquire the right to use the camera of candidate device 200 through collaborative connection and capture images through the camera of candidate device 200.

[0082] In the embodiments of this application, electronic device 100 can be directly connected to candidate device 200 via a near-field wireless communication connection or a local wired connection. For example, electronic device 100 and candidate device 200 may have one or more near-field communication modules selected from Wireless Fidelity (WiFi), Ultra Wide Bandwidth (UWB), Bluetooth, Near Field Communication (NFC), and ZigBee. Taking electronic device 100 as an example, electronic device 100 can use a near-field communication module (e.g., Bluetooth) to transmit signals to detect and scan nearby electronic devices, enabling electronic device 100 to discover nearby electronic devices (e.g., candidate device 200) via a near-field wireless communication protocol, establish a wireless communication connection with the nearby electronic devices, and transmit data to the nearby electronic devices.

[0083] In some embodiments, electronic device 100 and candidate device 200 can be connected to a local area network (LAN) via electronic device 300 using a wired or wireless fidelity (WiFi) connection. For example, electronic device 300 can be a third-party device such as a router, gateway, or smart device controller. In some embodiments, electronic device 100 and candidate device 200 can also be indirectly connected via at least one electronic device 400 in a wide area network (e.g., the Internet). For example, electronic device 400 can be a hardware server or a cloud server embedded in a virtualization environment. It is understood that through electronic device 300 and / or electronic device 400, electronic device 100 can indirectly establish wireless communication connections and data transmission with candidate device 200.

[0084] In some embodiments, electronic device 100 and candidate device 200 are electronic devices connected to the same local area network. In some embodiments, electronic device 100 and candidate device 200 are electronic devices logged into the same account; or, electronic device 100 and candidate device 200 are electronic devices logged into different accounts, but the different accounts (e.g., account 1 and account 2) are in a preset relationship, such as account 1 and account 2 being friends, account 1 authorizing account 2 to collaborative connections, or account 1 and account 2 being in the same family group. This application embodiment does not specifically limit these aspects.

[0085] It is understood that the structure shown in this embodiment does not constitute a specific limitation on the communication system 10. In other embodiments of this application, the communication system 10 may include more or fewer devices than those shown in the figure.

[0086] Figure 2 A schematic diagram of the structure of the electronic device 100 is shown.

[0087] Electronic device 100 may be a mobile phone, tablet computer (PAD), desktop computer, laptop computer, handheld computer, laptop, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The embodiments of this application do not impose any special restrictions on the specific type of electronic device.

[0088] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0089] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0090] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0091] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0092] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0093] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0094] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0095] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0096] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0097] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0098] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0099] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0100] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0101] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0102] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0103] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0104] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0105] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0106] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0107] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0108] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0109] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0110] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0111] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). Non-volatile memory can include disk storage devices and flash memory.

[0112] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0113] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0114] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0115] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0116] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0117] The audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal.

[0118] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0119] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0120] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Microphone 170C can be used to achieve functions such as sound signal acquisition, noise reduction, sound source identification, and directional recording.

[0121] The headphone jack 170D is used to connect wired headphones. The headphone jack 170D can be a USB 130 interface or other standard interfaces.

[0122] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194.

[0123] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 180B.

[0124] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0125] The accelerometer 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, it can detect the magnitude and direction of gravity.

[0126] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0127] The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED.

[0128] An ambient light sensor 180L is used to sense the ambient light intensity. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light intensity.

[0129] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0130] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature processing strategy.

[0131] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event.

[0132] The bone conduction sensor 180M can acquire vibration signals.

[0133] Button 190 includes the power button, volume buttons, etc. Button 190 can be a mechanical button or a touch button.

[0134] Motor 191 can generate vibration alerts.

[0135] Indicator 192 can be an indicator light, which can be used to indicate charging status, power changes, messages, notifications, etc.

[0136] The SIM card interface 195 is used to connect the SIM card.

[0137] The following describes the application scenarios of the device collaboration method provided in the embodiments of this application.

[0138] First, we will introduce several user interfaces for collaborative connections involved in the embodiments of this application.

[0139] For example, Figures 3A to 3D A schematic diagram of a collaborative connection interface for invoking an electronic device 100 is shown.

[0140] Figure 3A A main interface 11 on an electronic device 100 is shown, used to display applications installed on the electronic device 100. The main interface 11 may include: a status bar 101, a calendar indicator 102, a weather indicator 103, a tray 104 with icons of commonly used applications, and other application icons 105. Wherein:

[0141] The status bar 101 may include: one or more signal strength indicators for mobile communication signals (also known as cellular signals), operator name (e.g., "China Mobile"), one or more signal strength indicators for wireless fidelity (Wi-Fi) signals, battery status indicator, and time indicator.

[0142] The main interface 11 may also include a page indicator 106. Other application icons can be distributed across multiple pages, and the page indicator 106 can be used to indicate which page the user is currently viewing. Users can swipe left or right on the areas containing other application icons to view application icons on other pages.

[0143] like Figure 3A As shown, the electronic device 100 receives a swipe operation from the user on the status bar 101, and in response to the swipe operation, the electronic device 100 displays... Figure 3B The control center interface 12 shown includes shortcut icons 201 for several commonly used functions (e.g., ...). Figure 3B (The icons shown include WLAN, Bluetooth, NFC, and brightness adjustment bar).

[0144] like Figure 3B As shown, the control center interface 12 may also include a collaborative connection card 1, which displays a prompt message 202 and a control 203. The prompt message 202 can be used to inform the user that there are currently no available collaboratively connected devices, and that other devices can be searched by clicking the control 203.

[0145] In some embodiments, the control 203 may receive user input (e.g., a click operation), and in response to the input operation, the electronic device 100 displays... Figure 3C The collaborative connection interface 13 is shown. The collaborative connection interface 13 includes an icon 301 of the electronic device 100, icons of candidate devices for collaborative connection detected by the electronic device 100, and the names and / or models of the candidate devices. Each candidate device displayed in the collaborative connection interface 13 supports at least one collaborative capability.

[0146] For example, the icon of the electronic device 100 is located in the preset area 1 of the collaborative connection interface 13 (e.g. Figure 3C The circular area shown above), the icon of the aforementioned candidate device can be located within a preset area 2 surrounding the icon of the electronic device 100 (e.g., the circular area shown above). Figure 3C The circular area shown is included in the illustration. This application embodiment does not specifically limit the shape and position of the aforementioned preset area 1 and preset area 2. Subsequent embodiments will use the circular area shown in the illustration for preset area 1 and the circular area shown in the illustration for example. The icons for the aforementioned candidate devices may include an icon 302 for a tablet computer with model number AAA, an icon 303 for a laptop computer with model number YYY, an icon 304 for a speaker with model number ZZZ, an icon 305 for a projector with model number DDD, and an icon 306 for a printer with model number EEE.

[0147] In this embodiment, before displaying the collaborative connection interface 13, in response to an input operation (e.g., clicking) on ​​the control 203, the electronic device 100 detects candidate devices for collaborative connection and obtains relevant information for each candidate device. This relevant information may indicate some or all of the collaborative capabilities supported by the candidate device, including device identifier, device name, device model, and device type. Based on the relevant information of each candidate device, the electronic device 100 determines the icon of that candidate device and displays it in the collaborative connection interface 13. In one implementation, the electronic device 100 can detect electronic devices located on the same local area network (LAN) through signal detection and use them as candidate devices for collaborative connection. In another implementation, the electronic device 100 can query a server for electronic devices logged into the same account as itself and use the queried electronic devices as candidate devices for collaborative connection. The relevant information of the candidate devices may be requested by the electronic device 100 after detecting a candidate device, or it may be information that the electronic device 100 has already obtained and stored in previous collaborative connections with candidate devices.

[0148] In some embodiments, each candidate device icon in the collaborative connection interface 13 is additionally displayed with a capability badge corresponding to at least one capability that the candidate device can collaborate with. In one implementation, the aforementioned relevant information includes the collaborative capabilities that the candidate device can support, and the electronic device 100 can further determine the capability badge displayed on the candidate device icon based on the relevant information of the candidate device.

[0149] For example, see Figure 3D The collaborative connection interface 13 shown includes a tablet computer icon 302 with an additional display of the camera capability badge, a laptop computer icon 303 with an additional display of the audio input capability badge, a speaker icon 304 with an additional display of the audio output capability badge, a projector icon 305 with an additional display of the screen projection capability badge, and a printer icon 306 with an additional display of the printing capability badge.

[0150] In one implementation, the icons of candidate devices displayed on the collaborative connection interface 13 invoked by the electronic device 100 only show one capability badge. If a candidate device supports multiple collaborative capabilities, the aforementioned capability badge can be the capability badge corresponding to the capability that the candidate device has recently collaborated with, or it can be the capability badge corresponding to the capability that the candidate device collaborates with most frequently; no specific limitation is made here.

[0151] In some embodiments, the icons of candidate devices in the collaborative connection interface 13 rotate slowly relative to the icon of the electronic device 100.

[0152] In some embodiments, when the electronic device 100 does not detect a candidate device for collaborative connection... Figure 3C and Figure 3D The circular area of ​​the collaborative connection interface 13 shows icons of zero candidate devices. The electronic device 100 can display a prompt message in the collaborative connection interface 13 to inform the user that no candidate devices for collaborative connection have been detected.

[0153] In this embodiment of the application, under the same communication environment, the candidate devices displayed in the collaborative connection interface 13 that are accessed in different ways may be different, and the capability badges attached to the icons of the candidate devices may also be different.

[0154] In some embodiments, when the electronic device 100 is running application 1 in the foreground, it detects an input operation to bring up the collaborative connection interface 13; in response to the input operation, the electronic device 100 determines at least one preset capability required by application 1, detects candidate devices that can collaborate with the preset capability, and displays the icons of the candidate devices that can collaborate with the preset capability on the collaborative connection interface 13. In some embodiments, the icons of the candidate devices in the collaborative connection interface 13 are all accompanied by a capability badge displaying the preset capability.

[0155] For example, application 1 is a music application (APP). Figures 4A to 4D Another way to access the collaborative connection interface 13 is shown.

[0156] Electronic device 100 display Figure 4A When the music playback interface 14 of the music app shown is displayed, a swipe operation is received on the status bar; in response to the swipe operation, the electronic device 100 displays... Figure 4B The control center interface 12 shown is detected; an input operation is detected for the control 203 in the control center interface 12; in response to the input operation, the electronic device 100 confirms that the capabilities required by the music APP include audio output function, the electronic device 100 detects candidate devices that can cooperate with audio output function, and in Figure 4C or Figure 4D The collaborative connection interface 13 shown displays icons of candidate devices capable of collaborative audio output functionality. In some embodiments, see... Figure 4D The icons of the candidate devices all have a superscript label indicating their audio output capability. For example, electronic device 100 detects the following devices on the same local area network: a tablet computer of model AAA, a laptop computer of model YYY, a speaker of model ZZZ, a projector of model DDD, and a printer of model EEE; among them, the tablet computer, laptop computer, and speaker support collaborative connection with audio output capability, while other devices do not support collaborative connection with audio output capability; Figure 4C or Figure 4D The collaborative connection interface 13 shown includes an icon 302 for a tablet computer, an icon 303 for a laptop computer, and an icon 304 for a speaker.

[0157] Understandable. Figures 4A to 4D In the example, the input operation for bringing up the collaborative connection interface 13 may include: a swipe operation to bring up the control center interface 12, and a click operation on the control 203 in the control center interface 12.

[0158] In some embodiments, the electronic device 100 runs application 1 in the foreground. The user interface 1 of application 1 includes control 1, which is used to invoke the preset capabilities of available devices that have been coordinated. An input operation (e.g., a click) is detected on control 1. If there are currently no available devices with coordinated preset capabilities, in response to the input operation, the electronic device 100 displays control 2. An input operation (e.g., a click) is detected on control 2. The electronic device 100 detects candidate devices that can coordinate the preset capabilities and displays icons of these candidate devices in the coordination connection interface 13. In some embodiments, the icons of the candidate devices in the coordination connection interface 13 are all accompanied by a capability badge displaying the preset capabilities. It can be understood that the input operation that invokes the coordination connection interface 13 may include the input operation on control 1 and the input operation on control 2.

[0159] For example, application 1 is a music app. Figures 5A to 5D This illustrates another method for accessing the collaborative connection interface 13. User interface 1 is... Figure 5A The music playback interface 14 shown includes a switch control 311 (control 1), which is used to call the audio output function of the available, coordinated devices. Control 2 is... Figure 5B The control shown is 312.

[0160] Electronic device 100 display Figure 5A The music playback interface 14 of the music application shown includes a switching control 311. The electronic device 100 detects an input operation applied to the switching control 311. If no device with collaborative audio output functionality is currently available, in response to the input operation, the electronic device 100 displays a card 2 for collaborative connection. The card 2 includes a control 312 and a prompt message 313, which indicates that no device with collaborative audio output functionality is currently available, and the user can search for other devices. An input operation is detected for the control 312; in response to the input operation, the electronic device 100 detects candidate devices with collaborative audio output functionality and displays... Figure 5C or Figure 5DThe collaborative connectivity interface 13 shown displays icons of candidate devices capable of collaborative audio output functionality. In some embodiments, see [link to relevant documentation]. Figure 5D The icons for candidate devices all have a badge indicating their audio output capability.

[0161] In some embodiments, the electronic device 100 runs application 1 in the foreground; it detects an input operation by the user on application 1, which is used to execute a preset capability of application 1; if the electronic device 100 does not have the preset capability, in response to the input operation, the electronic device 100 displays control 2; it detects an input operation (e.g., a click operation) on control 2, the electronic device 100 detects candidate devices that can cooperate with the preset capability, and displays icons of the candidate devices that can cooperate with the preset capability in the collaborative connection interface 13. In some embodiments, the icons of the candidate devices in the collaborative connection interface 13 are all accompanied by a capability badge displaying the preset capability. It can be understood that the input operation that invokes the collaborative connection interface 13 may include: the input operation on application 1 and the input operation on control 2.

[0162] For example, application 1 is a Word app, and the preset capability is printing. If electronic device 100 detects an input operation that calls the printing function of the Word app, then after determining that the device does not have printing capabilities, electronic device 100 can display a card for collaborative connection and bring up the collaborative connection interface 13 through the card. Specifically, refer to the foregoing embodiments; further details will not be repeated here.

[0163] It should be noted that, Figures 3A to 5D Several methods for invoking the collaborative connection interface 13 have been exemplarily described. However, this application embodiment can also invoke the collaborative connection interface 13 in other ways, which are not specifically limited here. Furthermore, Figures 3A to 5D The collaborative connection interface 13 involved is an exemplary description of the embodiments of this application and should not be construed as limiting this application.

[0164] The following describes how to achieve collaborative connection between devices for specified capabilities through the collaborative connection interface.

[0165] This application provides five scenarios for implementing collaborative connections.

[0166] Scenario 1: The two electronic devices only support collaborative connection of one capability; Candidate device 200 supports at least one collaborative capability, and the icon of candidate device 200 can be attached with a capability badge of any of the above-mentioned capabilities (e.g., capability 1). When the icon of candidate device 200 is attached with the capability badge corresponding to capability 1, dragging the icon of candidate device 200 close to the icon of electronic device 100 or dragging the icon of electronic device 100 close to the icon of candidate device 200 can realize the collaborative connection of electronic device 100 to candidate device 200 for capability 1.

[0167] For example, with Figure 6A Taking the collaborative connection interface 13 shown as an example, Figures 6A to 7C The relevant user interface for the collaborative connection in Case 1 is shown.

[0168] like Figure 6A and Figure 6B As shown, in the collaborative connection interface 13, the tablet computer icon 302 displays a capability badge 501 corresponding to the camera. The tablet computer icon 302 can receive a user's swipe operation, with the swipe direction pointing towards the electronic device 100 icon 301. In response to the swipe operation, the electronic device 100 moves the tablet computer icon 302 towards its own icon 301. Figure 6B As shown, when icon 302 is detected to be dragged to the area where icon 301 is located, in response to the aforementioned sliding operation, electronic device 100 displays prompt information 401 and initiates a collaborative connection with the tablet computer's camera. Prompt information 401 is used to indicate that the device is in collaborative connection with the tablet computer's camera. In this embodiment, electronic device 100 may display prompt information 401 and initiate the collaborative connection when it detects icon 302 touching the preset area 1 where icon 301 is located, or it may trigger the display of prompt information 401 and the initiation of the collaborative connection only when it detects icon 302 touching icon 301; no specific limitation is made here.

[0169] In some embodiments, the electronic device 100 performs a collaborative connection to the camera of a tablet computer, including sending a collaborative connection request to the tablet computer for the camera. The collaborative connection request may include an identifier of the electronic device 100 and an identifier of the camera.

[0170] In some embodiments, such as Figure 6C As shown, after receiving a collaborative connection request from electronic device 100, the tablet computer displays a selection box, which includes a prompt message 402, a confirmation control 403, and a rejection control 404. The prompt message 402 indicates that a collaborative connection request for the camera has been received from electronic device 100.

[0171] like Figure 6CAs shown, the confirmation control 403 can receive user input. In response to the input, the tablet computer sends a collaborative connection response 1 to the electronic device 100, agreeing to the collaborative connection. After receiving the collaborative connection response 1 from the tablet computer, the electronic device 100 displays... Figure 6D The displayed message 404 changes the display status of the tablet's icon from a non-collaborating state to a collaborating state. Message 404 indicates that the device and the tablet's camera have established a collaborative connection. Figure 6D The "collaborated" state shown refers to the tablet computer's icon 302 being darker and attached to the device's icon 301. In this embodiment, the non-collaborated and collaborative states of the candidate device's icon are not specifically defined.

[0172] In some embodiments, the capability badges displayed alongside the candidate device's icon also include "not coordinated" and "coordinated" states. For example... Figure 6A As shown, before a successful camera collaboration connection, the camera's capability indicator 501 is in an uncollaborated state (e.g., not highlighted); as Figure 6D As shown, after a successful camera collaboration connection, the camera's capability icon 501 displays a collaborative state (e.g., highlighted). This embodiment does not specifically limit the non-collaboration and collaborative states of the capability icon.

[0173] In some embodiments, reference Figure 6E The control center interface 12 shown, after establishing a collaborative connection with the tablet's camera, displays icons of available collaborative devices (e.g., tablet icon 412) in the collaborative connection card 1. It can also display icons of uncollaborated candidate devices (e.g., laptop icon 413, speaker icon 414, and projector icon 415). The available device icon 412 is accompanied by a capability badge corresponding to the collaborative capability (i.e., camera). In this embodiment, the display state of icon 412 (e.g., darkened display) and / or prompt information 416 (e.g., the text "Collaborated") can indicate that the electronic device corresponding to the icon is a collaborative available device; the capability badge attached to icon 412 indicates the collaborative capability of the tablet. Furthermore, Figure 6E The card 1 shown may also include a control 411, which the user can use to call up... Figure 6D The collaborative connection interface 13 is shown.

[0174] like Figure 7A As shown, the rejection control 404 can receive user input. In response to the input, the tablet computer sends a rejection response 2 to the electronic device 100. Based on the rejection response 2, after confirming the failure of the collaborative connection, the electronic device 100 displays... Figure 7BThe system displays a prompt message 405 and redirects the tablet's icon 301 back to the preset area 2 (i.e., the circular area) where the candidate device is located. The prompt message 405 indicates that the camera connection between this device and the tablet has failed. In some embodiments, refer to... Figure 7C The control center interface 12 shown shows that after a failed connection with the tablet computer, the electronic device 100 displays the icons of the detected candidate devices in the connection card 1, so that the user can know which candidate devices are currently available for connection.

[0175] It should be noted that, in this embodiment, the failure of the collaborative connection is not limited to the user's input operation of rejecting the collaborative connection; other reasons may also cause the collaborative connection to fail. These other reasons are not specifically limited here. In one implementation, if the electronic device 100 does not receive a collaborative connection response 1 within a preset time after sending the collaborative connection request, the electronic device 100 determines that the collaborative connection has failed and displays... Figure 7B The collaborative connection interface 13 is shown.

[0176] In some embodiments, before the collaborative connection interface 13 receives the user's dragging icon swiping operation, the icons of the candidate devices (e.g., icon 302) all slowly rotate around the icon 301 of this device; when the collaborative connection interface 13 receives the user's dragging icon swiping operation, the icons of the candidate devices stop slowly rotating; during the collaborative connection process, the electronic device 100 can also play the connection animation; after determining that the collaborative connection is successful or failed, the electronic device 100 can terminate the above connection animation, and the icons of each candidate device resume rotating.

[0177] In some embodiments, after receiving a collaborative connection request from the electronic device 100, the tablet computer can decide whether to agree to the collaborative connection request based on a preset method, without needing to display anything. Figure 6CThe selection box shown is not specifically limited to the above-mentioned preset method in this application embodiment. In one implementation, before the electronic device 100 sends a collaborative connection request to the tablet computer, a verification information input box is also displayed. The verification information input box is used to input the verification information (e.g., password) for the collaborative connection of the tablet computer. The collaborative connection request may carry the verification information entered by the user in the verification information input box. If the tablet computer determines that the verification information is correct, it agrees to the collaborative connection request of the electronic device 100; otherwise, it rejects the collaborative connection request of the electronic device 100. In one implementation, if the tablet computer determines that it is logged into the same account as the electronic device 100, it agrees to the collaborative connection request of the electronic device 100; otherwise, it rejects the collaborative connection request of the electronic device 100. In one implementation, if the tablet computer determines that it is located on the same local area network as the electronic device 100, it agrees to the collaborative connection request of the electronic device 100; otherwise, it rejects the collaborative connection request of the electronic device 100.

[0178] In this embodiment of the application, the user can switch the capability badges displayed on the icon of the candidate device, and then establish a collaborative connection based on the switched capability badges.

[0179] For example, Figures 8A to 8D The user interface shows a user interface that allows switching the display of a tablet computer icon 302 with additional badges.

[0180] like Figure 8A and Figure 8B As shown, the icons of candidate devices in the collaborative connection interface 13 (e.g., the tablet icon 302) also display a capability badge 501 corresponding to the camera. Icon 302 can receive user input (e.g., a click). In response to this input, the electronic device 100 displays capability badges around icon 302 corresponding to collaborative capabilities other than the camera supported by the tablet, such as capability badge 502 for audio input and capability badge 503 for audio output. It is understood that if the tablet only supports a camera as a collaborative capability, then when the user clicks icon 302, no other capability badges will be displayed around icon 302.

[0181] like Figure 8B and Figure 8C As shown, capability badge 502 can receive user input (e.g., a click operation). In response to this operation, the capability badge 501 displayed on the tablet's icon 302 is switched to capability badge 502, and the capability badge 501 corresponding to the camera is displayed around the icon 302. Figure 8C and Figure 8D As shown, after the tablet computer's icon 302 receives user input (e.g., a touch operation), the electronic device 100 stops displaying capability badges around the icon 302. In some embodiments, Figure 8B After the capability indicator 502 receives the user's input, the electronic device 100 can also directly display it. Figure 8D The collaborative connection interface 13 shown is understandable. Figure 8D The tablet computer icon 302 shown has an additional badge 502 indicating audio input capability. Users can drag icon 302 to icon 301 on this device to enable the tablet computer's audio input function. Figure 8B and Figure 8C As shown, for the same ability badge (e.g., ability badge 501), the size of the ability badge 501 displayed around the icon 302 and the ability badge 501 additionally displayed on the icon 302 can be different.

[0182] For example, Figures 8E to 8G The user interface shows another option: the ability to switch the tablet's icon 302 to display a badge.

[0183] In response to Figure 8A The input operation (e.g., a click operation) of the icon 302 of the tablet computer is displayed on the electronic device 100. Figure 8E The collaborative connection interface 13 shown displays capability badges corresponding to all collaborative capabilities supported by the tablet computer around icon 302. The selected status indicates the capability badge currently displayed by icon 302. For example... Figure 8E As shown, the currently displayed capability badge 501 of icon 302 is in a selected state (e.g., highlighted), while the currently undisplayed capability badge 502 of icon 302 is in an unselected state (e.g., not highlighted).

[0184] like Figure 8E and Figure 8F As shown, capability badge 502 can receive user input (e.g., a click). In response to this operation, capability badge 501 changes from a selected state to an unselected state, and capability badge 502 changes from an unselected state to a selected state. That is, the capability badge 501 displayed on the tablet computer icon 302 is switched to capability badge 502. Figure 8F and Figure 8G As shown, after the tablet computer's icon 302 receives user input (e.g., a touch operation), the electronic device 100 stops displaying capability badges around the icon 302. In some embodiments, Figure 8E After the capability indicator 502 receives the user's input, the electronic device 100 can also directly display it. Figure 8G The collaborative connection interface 13 is shown.

[0185] In this embodiment, users can also achieve collaborative connection of the capability corresponding to the capability by dragging the capability badges around the icon 302 to the area where the icon 301 of the electronic device 100 is located.

[0186] For example, with Figure 9A Taking the collaborative connection interface 13 shown as an example, as Figure 9A and Figure 9B As shown, the user drags the audio input function capability badge 502 to the area where icon 301 is located (i.e., the aforementioned preset area 1, or the display area of ​​icon 301) using a swipe gesture and then releases the swipe; Figure 9C As shown, in response to a sliding operation on capability index 502, electronic device 100 moves capability index 502 along the sliding direction of the sliding operation; as Figure 9C As shown, when the capability badge 502 is dragged to the area where the icon 301 is located and the user's finger leaves the screen, the electronic device 100 displays the tablet computer icon 302 with the additional capability badge 502 in the preset area 1 (i.e., the aforementioned circular area), along with a prompt message 406. The prompt message 406 indicates that the device and the tablet computer are in collaborative connection for audio input function. For more details, please refer to the following section. Figures 6B to 7C The details regarding collaborative connections will not be elaborated upon here.

[0187] In addition, regarding Figures 3A to 5D The collaborative connection interface 13 shown here does not have attached capability badges. Users can also select to attach capability badges to the icons of candidate devices. For details, please refer to [link / reference needed]. Figures 8A to 8F For related descriptions and information on how to achieve collaborative connectivity after adding a capability badge to the candidate device's icon, please refer to [link / reference]. Figures 6A to 7C as well as Figures 9A to 9C The relevant descriptions will not be repeated here.

[0188] In some embodiments, the collaborative connection of capabilities corresponding to the additional capability badges displayed by the candidate device 200 can also be achieved by dragging the icon 301 of this device to the area where the icon of the candidate device 200 is located.

[0189] For example, with Figure 10A Taking the collaborative connection interface 13 shown as an example, Figures 10A to 10F The relevant user interface for collaborative connectivity in this embodiment is shown.

[0190] like Figure 10A and Figure 10BAs shown, in the collaborative connection interface 13, the tablet computer icon 302 additionally displays a capability badge 501 corresponding to the camera. The device icon 301 can receive a user's swipe operation, the swipe direction of which points towards the tablet computer icon 302; in response to the swipe operation, the electronic device 100 moves the icon 301 towards the icon 302. Figure 10C As shown, when icon 301 touches icon 302, in response to the above-mentioned sliding operation, electronic device 100 displays prompt information 401 and performs a collaborative connection to the camera of the tablet computer.

[0191] like Figure 10D As shown, when the collaborative connection is confirmed to be successful, the electronic device 100 displays a prompt message 404 and changes the display status of the tablet's icon from a non-collaboration state to a collaborative state. When the collaborative connection is confirmed to be unsuccessful, the electronic device 100 displays... Figure 10E The system displays message 405 and then displays the tablet icon 302 back in the circular area where the candidate device is located.

[0192] Figure 10D The "collaborated" state shown refers to the tablet's icon 302 being darker and attached to the device's icon 301. This embodiment does not specifically limit the "non-collaborated" and "collaborated" states of the icon. For example, when a successful collaboration connection is determined, the electronic device 100 displays... Figure 10F The collaborative connection interface shown is as follows. Figure 10F Another cooperative state of icon 302 is shown, that is, icon 302 is darkened and bounces back to the preset area 2 where the candidate device is located (i.e. the ring area shown in the figure); the non-cooperative state of other candidate devices in the ring area is shown as no darkening of color.

[0193] In this embodiment of the application, after the electronic device 100 establishes a cooperative connection with the capability 1 of the candidate device 200, the user can cancel the cooperative connection with the capability 1 of the candidate device 200.

[0194] The following explanation uses a tablet computer's camera as an example. In some embodiments, see [link to relevant documentation]. Figure 10D and Figure 10F The tablet computer's "collaborated" icon 302 also displays a camera capability badge; the tablet computer's "collaborated" icon 302 can receive a user's cancellation operation 1; in response to the cancellation operation 1, the electronic device 100 can cancel the collaborative connection with the tablet computer's camera and switch the display state of icon 302 to a "non-collaborated" state. In this embodiment, the cancellation operation 1 is not specifically limited, and the cancellation operation 1 corresponding to different "collaborated" states may be different.

[0195] For example, Figure 11AThe icon 302, which is in a cooperative state, is attached to the icon 301 of this device. The aforementioned cancellation operation 1 may include dragging the icon 302 away from the icon 301 of this device by a sliding operation on the icon 302 and then releasing it. Figure 11B and Figure 11C As shown, when the above-mentioned cancellation operation 1 is detected, the electronic device 100 displays a prompt message to cancel the collaborative connection and cancels the collaborative connection with the tablet's camera. Figure 11D As shown, after canceling the collaborative connection, the display state of icon 302 is switched from the collaborative state to the non-collaborative state, and icon 302 is bounced back to the circular area. In one implementation, when the user performs the above-mentioned cancellation operation 1, the electronic device 100 can also directly display... Figure 11D The collaborative connection interface shown does not need to display the above prompt information.

[0196] For example, refer to Figure 10F The icon 302, which is in a coordinated state, is displayed in the circular area. The cancellation operation described above can include long-pressing the icon 302. The cancellation operation can also include gestures such as double-tapping; this application does not impose specific limitations.

[0197] In some embodiments, when the collaborative connection interface 13 receives a user's input operation to cancel the collaborative connection (e.g., the cancellation operation 1 mentioned above), the icons of the candidate devices stop rotating slowly; during the process of the electronic device 100 canceling the collaborative connection, the electronic device 100 may also play a cancellation animation; when it is determined that the collaborative connection has been canceled, the electronic device 100 may terminate the cancellation animation, and the icons of each candidate device resume rotating.

[0198] Scenario 2: Electronic device 100 and candidate device 200 support collaborative connection with multiple capabilities; candidate device 200 supports at least one collaborative capability, and the icon of candidate device 200 displays capability badges for one or more of the aforementioned capabilities (e.g., capability 1 and capability 2). When the icon of candidate device 200 displays capability badges corresponding to capability 1 and capability 2, dragging the icon of candidate device 200 close to the icon of electronic device 100 or vice versa enables collaborative connection of capability 1 and capability 2 between electronic device 100 and candidate device 200.

[0199] In the second scenario, collaborative connections for a single capability can be referenced as described above. Figures 6A to 7C as well as Figures 9A to 10F The relevant descriptions will not be repeated here.

[0200] For example, with Figure 12A Taking the collaborative connection interface 13 shown as an example, Figures 12A to 12G The user interface for adding multiple capability badges is shown.

[0201] In some embodiments, users can add capability badges corresponding to multiple capabilities to the tablet's icon 302. For example... Figure 12A and Figure 12B As shown, in the collaborative connection interface 13, the tablet computer icon 302 currently displays a capability badge 502 corresponding to the audio output function. The icon 302 can receive user input (e.g., a click). In response to the input, the electronic device 100 displays capability badges corresponding to all collaborative capabilities supported by the tablet computer around the icon 302, such as capability badge 501 for the camera, capability badge 502 for the audio input function, and capability badge 503 for the audio output function. Figure 12B As shown, the currently displayed capability badge 502 of icon 302 is in a selected state (e.g., highlighted), while the currently undisplayed capability badge 501 of icon 302 is in an unselected state (e.g., not highlighted).

[0202] Users can toggle the display status of ability badges from selected to unselected. For example... Figure 12B and Figure 12C As shown, users can switch the display state of ability badge 501 to selected state, adding ability badge 501 as an attached ability badge to icon 302. Similarly, as... Figure 12C and Figure 12D As shown, users can also switch the display status of ability badge 503 to the selected state, adding ability badge 503 as an additional ability badge displayed alongside icon 302. For example... Figure 12D and Figure 12E As shown, the selected capability badge 502 can receive user input (e.g., a click operation); in response to the input operation, the display state of capability badge 502 is switched to the unselected state, and the display of capability badge 502 on icon 302 is stopped.

[0203] In some embodiments, such as Figure 12E and Figure 12F As shown, after the user selects additional display capability badges (i.e. capability badges 501 and 503), icon 302 can receive user input operations (e.g., touch operations). In response to the input operation, electronic device 100 stops displaying capability badges around icon 302.

[0204] In some embodiments, Figures 12B to 12E During the capability icon selection process, when the user selects a capability icon (i.e., toggles the display state of that capability icon to the selected state), icon 302 may not display that capability icon. After the user completes the selection, electronic device 100... Figure 12FOnly then did China begin attaching a display capability badge to icon 302. For example, as shown... Figure 12G As shown, when the user selects the ability badges 501 and 503 displayed around the icon 302, the icon 302 does not need to display the above two ability badges.

[0205] For example, Figures 13A to 13J The relevant user interface for implementing collaborative connections in scenario two is shown.

[0206] like Figure 13A As shown, the icons 302 of the additional capability badges 501 and 503 can receive user swipe operations, with the swipe direction pointing towards icon 301 on the electronic device 100. When icon 302 is detected to be in contact with the area where icon 301 is located, in response to the user's swipe operation, the electronic device 100 displays a prompt message 601 and performs a collaborative connection with the camera and audio output functions of the tablet computer. The prompt message 601 is used to indicate that the device is in collaborative connection with the camera and audio output functions of the tablet computer.

[0207] In some embodiments, the electronic device 100 performs a cooperative connection to the camera and audio output functions of a tablet computer, which may include: the electronic device 100 sending a cooperative connection request to the tablet computer for the camera and audio output functions.

[0208] In some embodiments, such as Figure 13B As shown, after receiving a collaborative connection request from electronic device 100, the tablet computer displays a selection box. The selection box includes a prompt message 602, a camera selection control 603, an audio output function selection control 604, a confirmation control 605, and a rejection control 606. The prompt message 602 indicates that a collaborative connection request for the camera and audio output functions has been received from electronic device 100.

[0209] like Figure 13C As shown, after the camera selection control 603 and the audio output function selection control 604 receive the user's selection operation (e.g., a click operation), the confirmation control 605 can receive the user's input operation. In response to the above input operation, the tablet computer sends a collaborative connection response 3 to the electronic device 100. The collaborative connection response 3 is used to indicate agreement to the collaborative connection of the camera and audio output functions. After receiving the collaborative connection response 3 sent by the tablet computer, the electronic device 100 displays... Figure 13D The system displays a notification message 607 and changes the display status of the tablet's icon from non-collaboration to collaborative. The collaborative icon 412, along with the capability indicator and notification message 607, indicates that this device has established a collaborative connection with the tablet's camera and audio output function.

[0210] In some embodiments, reference Figure 13E The control center interface 12 shown shows that after establishing a collaborative connection for the camera and audio output functions of the tablet computer, the electronic device 100 displays the icons of the available collaborative devices (e.g., the tablet computer icon 412) in the collaborative connection card 1, and the tablet computer icon 412 also displays capability badges corresponding to the camera and audio output functions; the capability badges displayed on the icon 412 indicate that a collaborative connection has been established with the camera and audio output functions of the tablet computer.

[0211] In some embodiments, such as Figure 13F As shown, after the camera selection control 603 receives the user's selection operation (e.g., a click operation), the confirmation control 605 can receive the user's input operation. In response to the input operation, the tablet computer sends a collaborative connection response 4 to the electronic device 100. The collaborative connection response 4 is used to indicate whether to agree to the collaborative connection of the camera and to reject the collaborative connection of the audio output function. After receiving the collaborative connection response 4, the electronic device 100 displays... Figure 13G The displayed message 608 changes the display status of the tablet's icon 302 from a non-collaborating state to a collaborating state, and stops attaching a capability badge for audio output function to icon 302. Message 607 indicates that a collaborative connection has been established between this device and the tablet's camera, and that a collaborative connection with the tablet's audio output function has failed.

[0212] Compared to Figure 13E The image shows card 1 in the collaborative connection diagram. The camera collaborative connection is successful, but the audio output function collaborative connection fails. Figure 13H In the collaborative connection card 1 shown, the icon 412 of the collaborative tablet only has an additional display of the capability badge corresponding to the camera.

[0213] like Figure 13I As shown, the rejection control 404 can receive user input. In response to the input, the tablet computer sends a rejection response 5 to the electronic device 100. After receiving the rejection response 5 from the tablet computer, the electronic device 100 displays... Figure 13J The system displays message 609 and displays the tablet's icon 301 back in the circular area where the candidate device is located. Message 609 indicates that the connection between this device and the tablet's camera and audio output functions has failed.

[0214] In some embodiments, after the electronic device 100 establishes a collaborative connection with at least one capability of the candidate device 200 (e.g., a tablet computer), the user can also select other capabilities of the candidate device for collaborative connection.

[0215] For example, Figures 14A to 14E The user interface for continuing collaborative connections between already collaborating tablets is shown.

[0216] like Figure 14A As shown, the electronic device 100 has established a cooperative connection with the audio input function of the tablet computer, and the icon 302 of the tablet computer in the cooperative state also displays the capability badge 502 corresponding to the audio input function. Figure 14A and Figure 14B As shown, icon 302 can receive user input operations (such as click operations). In response to the above input operations, electronic device 100 displays capability badges corresponding to the collaborative capabilities supported by the tablet computer around icon 302, such as capability badge 502 corresponding to audio output function and capability badge 501 corresponding to camera.

[0217] like Figure 14B and Figure 14C As shown, taking capability icon 501 as an example, capability icon 501 can receive user input operations (e.g., click operations); in response to the above input operation, electronic device 100 stops displaying capability icons around icon 302, displays a prompt message 610 indicating that a collaborative connection is in progress, and performs a collaborative connection with the tablet computer's camera. In some embodiments, such as Figure 14C As shown, in response to the above input operation, the electronic device 100 also adds a display camera capability badge 501 to the icon 302.

[0218] like Figure 14D As shown, after the electronic device 100 confirms a successful collaborative connection, it displays a prompt message 611 indicating that the collaborative connection has been successfully established. In some embodiments, such as... Figure 14C As shown, before a successful camera collaboration connection, the camera's capability badge 501, displayed on icon 302, is in an uncollaborated state (e.g., not highlighted); as Figure 14D As shown, after confirming that the camera collaboration connection is successful, the electronic device 100 will also switch the camera's capability badge 501 to the collaboration state (e.g., deepen the display).

[0219] like Figure 14E As shown, after determining that the camera's collaborative connection has failed, the electronic device 100 displays a prompt message 612 indicating that the collaborative connection has failed, and stops displaying the camera's capability badge 501 on the icon 302 of the collaborative state.

[0220] In some embodiments, when multiple capability badges are displayed alongside the icon of the candidate device 200 (e.g., the icon 302 of a tablet computer) in the ring-shaped area, the capabilities corresponding to the multiple capability badges can also be collaboratively connected by dragging the icon 301 of this device to the area where the icon of the candidate device 200 is located. Specifically, please refer to... Figures 10A to 10F The relevant descriptions will not be repeated here.

[0221] In this embodiment, after the electronic device 100 establishes a cooperative connection with multiple capabilities (e.g., capability 1 and capability 2) of the candidate device 200, the user can cancel the cooperative connection with the candidate device 200 for the aforementioned multiple capabilities or one of the aforementioned capabilities. The following description uses a tablet computer as an example.

[0222] In some embodiments, see Figure 14D The tablet computer's "collaborated" icon 302 also displays capability badges for the camera and audio output functions. The tablet computer's "collaborated" icon 302 can receive a user's cancellation operation 1; in response to the cancellation operation 1, the electronic device 100 can cancel the collaborative connection of all collaborative capabilities (i.e., camera and audio output functions) of the tablet computer and switch the display state of icon 302 to the "uncollaborated" state. Cancellation operation 1 can be referred to the relevant description in Case 1, and will not be repeated here.

[0223] In some embodiments, the icon 302 of the tablet computer indicating the collaborative status is appended with multiple capability badges, allowing the user to individually cancel the collaborative connection of one of the multiple collaborative capabilities. For example, using... Figure 15A Taking the already coordinated state as an example, Figures 15A to 15D The relevant user interface for independently canceling the collaborative connection of the tablet's camera is shown.

[0224] like Figure 15A As shown, the tablet's collaborative icon 302 can also receive user input (e.g., a click); such as Figure 15B As shown, in response to the above input operation, capability badges corresponding to all collaborating capabilities of the tablet are displayed. These capability badges include those corresponding to collaborating capabilities (e.g., camera capability badge 501 and audio input capability badge 502) and those corresponding to non-collaborating capabilities (e.g., audio output capability badge 503). Collaborating capability badges (e.g., capability badge 501) are selected, while non-collaborating capability badges (e.g., capability badge 503) are unselected. Figure 15C As shown, capability icon 501 can receive user input (e.g., a click operation). In response to this input operation, electronic device 100 can cancel the collaborative connection with the tablet computer's camera and display a prompt message indicating the cancellation of the collaborative connection; as shown... Figure 15D As shown, after canceling the collaborative connection to the tablet's camera, the icon of the tablet that was in a collaborative state will stop displaying the camera capability badge 501.

[0225] In the embodiments of this application, in the aforementioned Case 1 and Case 2, the electronic device 100 can support establishing collaborative connections with multiple devices simultaneously.

[0226] For example, Figures 16A to 16D The user interface for establishing collaborative connections between electronic device 100 and two other devices is shown.

[0227] In some embodiments, electronic device 100 can simultaneously establish collaborative connections with the same capability of multiple candidate devices.

[0228] like Figure 16A and Figure 16B As shown, the electronic device 100 has established a collaborative connection with the audio output function and camera of the tablet computer; the icon 303 of the laptop computer displays a capability badge 501 corresponding to the camera; the user can drag the icon 303 to the area where the icon 301 of the electronic device 100 is located by sliding; when the icon 302 is detected to be dragged to the area where the icon 301 is located, in response to the above sliding operation, the electronic device 100 displays a prompt message 616 indicating that a collaborative connection is in progress, and performs a collaborative connection with the camera of the laptop computer.

[0229] like Figure 16C As shown, when the collaborative connection to the laptop's camera is confirmed to be successful, a prompt message 617 indicating successful collaborative connection is displayed, and the display status of the laptop's icon 303 changes from a non-collaborative state to a collaborative state; as shown... Figure 16D As shown, when it is determined that the collaborative connection to the laptop's camera has failed, a prompt message 618 indicating that the collaborative connection has failed is displayed, and the laptop's icon 303 is bounced back to the circular area where the candidate device is located.

[0230] For details, please refer to Figures 6A to 7C The relevant descriptions will not be repeated here.

[0231] In some embodiments, for the same capability, electronic device 100 may establish a cooperative connection with at most one device. Figure 16D The reason for the failure of collaborative connection may be that it is unable to simultaneously connect to the cameras of multiple devices. Electronic device 100 can... Figure 16D The system displays message 618 indicating that it cannot simultaneously connect to cameras from multiple devices.

[0232] Scenario 3: Electronic device 100 and candidate device 200 support at most one type of collaborative connection; candidate device 200 supports at least one collaborative capability, and the icon of electronic device 100 can be attached with a capability badge of any of the above-mentioned capabilities (e.g., capability 1). In some embodiments, when the icon of electronic device 100 is attached with a capability badge corresponding to capability 1, dragging the icon of candidate device 200 close to the icon of electronic device 100 or dragging the icon of electronic device 100 close to the icon of candidate device 200 can realize the collaborative connection of capability 1 between electronic device 100 and candidate device 200.

[0233] For example, with Figure 17A Taking the collaborative connection interface 13 shown as an example, Figures 17A to 17B The relevant user interface for attaching a capability badge to icon 301 of electronic device 100 is shown.

[0234] like Figure 17A and Figure 17B As shown, the icon 301 of the electronic device 100 can receive user input (e.g., a click operation). In response to the input operation, the electronic device 100 displays at least one capability badge around the icon 301. In one implementation, the at least one capability badge includes all collaborative capabilities supported by candidate devices displayed in the collaborative connection interface 13. Figure 17B As shown, at least one capability icon includes a camera capability icon 501, an audio output capability icon 503, an audio input capability icon 502, a screen projection capability icon 504, and a printing capability icon 505.

[0235] In some embodiments, such as Figure 17B As shown, the capability marker 503 corresponding to the audio output function can receive user input (e.g., click operations); as Figure 17C As shown, in response to the above input operation, the electronic device 100 switches the display state of the capability badge 503 to the selected state and adds the display capability badge 503 to the icon 301. Figure 17C The icon 301 shown can receive user input; for example Figure 17D As shown, in response to the aforementioned input operation, the electronic device 100 stops displaying capability badges around icon 301. In one implementation, in response to... Figure 17B The input operation indicated by capability marker 503 can be directly displayed by electronic device 100. Figure 17D The collaborative connection interface 13 is shown.

[0236] In some instances, the user clicks Figure 17BFollowing the capability badge 503 shown, the electronic device 100 can display only candidate device icons in the circular area that can coordinate with the capability corresponding to that capability badge. For example, printers and projectors do not support coordinateable audio output functionality; such as... Figure 17E As shown, different Figure 17C The collaborative connection interface 13 shown, in response to an input operation on capability icon 503, stops the display of printer icon 306 and projector icon 305 in the circular area of ​​the electronic device 100. (As shown) Figure 17E and Figure 17F As shown, icon 301 can receive user input. In response to the input, electronic device 100 stops displaying capability badges around icon 301. In one implementation, in response to... Figure 17B The input operation indicated by capability marker 503 can also be directly displayed on the electronic device 100. Figure 17F The collaborative connection interface 13 is shown.

[0237] For instructions on how to switch the capability badges displayed on the electronic device's 100 additional display in scenario three, please refer to [link / reference]. Figures 8A to 8G The implementation method for switching capability badges will not be elaborated here.

[0238] For example, with Figure 18A Taking the collaborative connection interface 13 shown as an example, Figures 18A to 18C The relevant user interface for a collaborative connection of one of the capabilities in Case 3 is shown.

[0239] like Figure 18A As shown, after the user adds a capability badge 503 to the icon 301 of this device, the user can drag the icon 302 to the area where the icon 301 is located by sliding. When the device detects that the icon 302 is in contact with the icon 301 or the preset area 1 where the icon 301 is located, in response to the above sliding operation, the electronic device 100 displays a prompt message 701 indicating that it is in collaborative connection, and performs collaborative connection with the audio output function of the tablet computer.

[0240] like Figure 18B As shown, when the audio output function is successfully connected, the electronic device 100 displays a prompt message 702 indicating successful connection, switches icon 302 from a non-connected state to a connected state, and adds a capability badge 503 for the audio output function to icon 302. In one implementation, as... Figure 18B As shown, after a successful collaborative connection, the electronic device 100 stops displaying the capability badge 503 on its own icon 301; and displays all detected non-collaborative candidate devices in the circular area of ​​the candidate devices.

[0241] like Figure 18CAs shown, when it is determined that the audio output function collaborative connection has failed, the electronic device 100 displays a prompt message 703 indicating that the collaborative connection has failed, and the icon 302 is bounced back to the circular area where the candidate device is located.

[0242] Furthermore, for how to cancel the collaborative connection of a collaborative capability in Scenario 3, please refer to the relevant description in Scenario 1, which will not be repeated here. It should be noted that, taking the tablet's camera as an example, if the collaborative connection to the tablet's camera is canceled in Scenario 3, the tablet's icon 302 will no longer display the camera capability badge after the collaborative connection is canceled.

[0243] In some embodiments, the ability corresponding to the ability badge displayed on the additional display of the icon 302 can also be linked by dragging the icon 301 of this device to the area where the icon 302 of the tablet computer is located.

[0244] For example, with Figure 19A Taking the collaborative connection interface 13 shown as an example, Figures 19A to 19E The relevant user interface for collaborative connectivity in this embodiment is shown.

[0245] like Figure 19A and Figure 19B As shown, the user can drag the device icon 301 to the tablet icon 302 by sliding. When the device detects that the icon 301 is touching the tablet icon 302, in response to the sliding operation, the electronic device 100 displays a prompt message 701 indicating that a collaborative connection is in progress, and performs a collaborative connection with the audio output function of the tablet.

[0246] like Figure 19C As shown, when the audio output function is successfully connected, the electronic device 100 displays a prompt message 702 indicating successful connection, switches the tablet computer icon 302 from non-connected to connected state, adds a capability badge 503 for the audio output function to the icon 302, and pops the device icon 301 back into the circular area. Figure 19D This illustrates the coordinated state of another icon 302 provided in an embodiment of this application, such as... Figure 19D As shown, the icon 302 in the collaborative state is highlighted and bounces back into the circular area along with the device's icon 301. In one implementation, as... Figure 19C and Figure 19D As shown, after icon 301 bounces back to the circular area, the display of capability badges stops.

[0247] like Figure 19EAs shown, when it is determined that the audio output function has failed to connect, the electronic device 100 displays a prompt message 703 indicating that the connection has failed, and the device icon 301 is pushed back to the circular area. The icon 301 can continue to display the capability badge 503 of the audio output function.

[0248] Similar to Figures 9A to 9C In scenario three, users can also drag the capability badges (e.g., capability badge 503) displayed around the device's icon 301 to the tablet's icon 302 to achieve collaborative connection to the audio output function corresponding to capability badge 503.

[0249] For example, such as Figure 19F and Figure 19G As shown, users can drag the capability badge 503 to the tablet icon 302 using a swipe gesture; Figure 19H As shown, when the capability badge 503 is detected to be in contact with the icon 302 and the user releases their hand, the electronic device 100 displays the above-mentioned sliding operation. Figure 19H The system displays prompt message 701 and performs a collaborative connection with the audio output function of the tablet computer.

[0250] like Figure 19I As shown, when the audio output function is successfully connected, the electronic device 100 displays a prompt message 702 indicating that the connection is successful, switches the tablet computer icon 302 from the non-connected state to the connected state, and adds a capability badge 503 for the audio output function to the icon 302.

[0251] like Figure 19J As shown, when it is determined that the audio output function has failed to connect, the electronic device 100 displays a prompt message 703 indicating that the connection has failed, and the icon 301 may also display a capability badge 503 for the audio output function.

[0252] Scenario 4: Electronic device 100 and candidate device 200 support collaborative connection with multiple capabilities; candidate device 200 supports at least one collaborative capability, and the icon of electronic device 100 can be attached with a capability badge displaying one or more of the above-mentioned capabilities (e.g., capability 1 and capability 2).

[0253] In some embodiments, when the icon of the electronic device 100 is attached with capability badges corresponding to capability 1 and capability 2, dragging the icon of the candidate device 200 close to the icon of the electronic device 100 or dragging the icon of the electronic device 100 close to the icon of the candidate device 200 can enable the electronic device 100 to achieve collaborative connection of capability 1 and capability 2 of the candidate device 200.

[0254] For scenario four, collaborative connections for a single capability can refer to the aforementioned examples. Figures 18A to 19JThe relevant descriptions will not be repeated here.

[0255] For example, with Figure 20A Taking the collaborative connection interface 13 shown as an example, Figures 20A to 20D The relevant user interface is shown with two capability badges attached to the icon 302 of the electronic device 100.

[0256] like Figure 20A and Figure 20B As shown, the icon 301 of the electronic device 100 additionally displays a capability badge 503 for audio output functionality; the icon 301 can receive user input (e.g., a click operation), and in response to the input operation, the electronic device 100 displays at least one capability badge around the icon 301. In one implementation, the at least one capability badge includes collaborative capabilities supported by the candidate device currently displayed in the collaborative connection interface 13. Compared to Figure 17B , Figure 20B The candidate devices in the list do not support printing functionality, and at least one of the above capability subscripts does not include the capability subscript 505 for printing functionality.

[0257] In some embodiments, such as Figure 20B As shown, the capability label 501 corresponding to the camera can receive user input (such as a click operation); as Figure 20C As shown, in response to the above input operation, the electronic device 100 adds capability badge 501 as an additional capability badge displayed on the device's icon 301. In one implementation, in response to the above input operation, the electronic device 100 also stops displaying icons of candidate devices that do not support the camera corresponding to capability badge 501 (e.g., speaker icon 304), and / or switches the surrounding capability badges 501 from an unselected state to a selected state. Figure 20C The icon 301 shown can receive user input (such as a click); for example Figure 20D As shown, in response to the aforementioned input operation, the electronic device 100 stops displaying at least one of the aforementioned capability badges around the icon 301. In one implementation, in response to... Figure 20B The input operation indicated by capability marker 503 can be directly displayed by electronic device 100. Figure 20D The collaborative connection interface 13 is shown.

[0258] For instructions on how to switch the additional capability badges of electronic devices in scenario four, please refer to [link / reference]. Figures 12A to 12F The implementation method of switching the tablet's icon 302 with the attached capability badge is not described here.

[0259] For example, with Figure 21A Taking the collaborative connection interface 13 shown as an example, Figures 21A to 21DThe relevant user interface for the collaborative connection of the two capabilities in Case 4 is shown.

[0260] like Figure 21A As shown, after the user adds capability badges 503 and 501 to the icon 301 of this device, the user can drag the icon 302 to the area where the icon 301 is located by sliding. When the device detects that the icon 302 is in contact with the icon 301 or the preset area 1 where the icon 301 is located, the electronic device 100 displays a prompt message 711 indicating that it is in collaborative connection, and performs collaborative connection with the audio output function and camera of the tablet computer.

[0261] like Figure 21B As shown, when it is determined that both the audio output function and the camera have successfully connected, the electronic device 100 displays a prompt message 712 indicating successful connection, switches icon 302 from a non-connected state to a connected state, and adds a capability badge 501 for the camera and a capability badge 503 for the audio output function to icon 302. In one implementation, after successful connection, icon 301 of this device stops displaying the audio output function icon 503 and the camera icon 501, and displays all detected non-connected candidate devices in a circular area. Figure 21B As shown, the electronic device 100 displays the speaker icon 304, the projector icon 305, and the printer icon 306 again in the circular area.

[0262] like Figure 21C As shown, when the camera collaboration connection is confirmed to be successful, but the audio output function collaboration connection fails, the electronic device 100 displays a prompt message 713, switches icon 302 from a non-collaboration state to a collaboration state, and attaches a camera capability badge 501 to icon 302. The prompt message 713 indicates that the camera collaboration connection with the tablet computer is successful, but the audio output function collaboration connection with the tablet computer has failed. In one implementation, after the camera collaboration connection is successful but the audio output function collaboration connection fails, the device's icon 301 stops displaying the camera icon 501, and displays detected candidate devices that support audio output function but are not collaborating in a circular area. Figure 21C As shown, the electronic device 100 displays the speaker icon 304 again in the circular area.

[0263] like Figure 21D As shown, when it is determined that both the audio output function and the camera have failed to connect, the electronic device 100 displays a prompt message 714 indicating that the connection has failed, and the icon 302 is bounced back to the circular area where the candidate device is located.

[0264] Similar to Figures 15A to 15DIn scenario four, when the device icon 301 displays multiple capability badges, the collaborative connection of the capabilities corresponding to these multiple capability badges can be achieved by dragging the device icon 301 to the area where the candidate device icon 200 is located. For details, please refer to... Figures 19A to 19E The relevant descriptions will not be repeated here.

[0265] In some embodiments, after the electronic device 100 establishes a collaborative connection with at least one capability of the candidate device 200 (e.g., a tablet computer), the user can also select other capabilities of the candidate device for collaborative connection.

[0266] For example, Figures 22A to 23C The user interface for continuing collaborative connections between already collaborating tablets is shown.

[0267] like Figure 22A As shown, electronic device 100 has now established a collaborative connection with the audio output function of the tablet computer. (As indicated...) Figure 22B , Figure 22C and Figure 22D As shown, taking ability badge 501 as an example, users can select ability badge 501 to add it as an additional ability badge displayed alongside icon 301. For details, please refer to... Figures 20B to 20D The relevant descriptions will not be repeated here.

[0268] like Figure 22D and Figure 23A As shown, after the user adds a capability badge 501 to the icon 301 of this device, the user can drag the icon 301 of this device to the area where the icon 302 is located by sliding. When the device detects that the icon 301 touches the icon 302, the electronic device 100 displays a prompt message 721 in the collaborative connection prompt and performs a collaborative connection with the camera of the tablet computer.

[0269] like Figure 23B As shown, when the collaborative connection is confirmed to be successful, the electronic device 100 displays a prompt message 722 indicating that the collaborative connection is successful, adds a display camera capability badge 501 to the icon 302 that is in a collaborative state, and pops the device icon 301 back into the circular area 1.

[0270] like Figure 23C As shown, after determining that the camera's collaborative connection has failed, the electronic device 100 displays a prompt message 723 indicating that the collaborative connection has failed, and bounces the device's icon 301 back into the circular area.

[0271] For more details, you can refer to... Figures 19A to 19E The relevant descriptions will not be repeated here.

[0272] Furthermore, for how to cancel the collaborative connection of a collaborative capability in Scenario 4, please refer to the relevant description in Scenario 2, which will not be repeated here. It should be noted that, taking the tablet's camera as an example, if the collaborative connection to the tablet's camera is canceled in Scenario 4, the tablet's icon 302 will no longer display the camera capability badge after the collaborative connection is canceled.

[0273] In the embodiments of this application, in the aforementioned situations three and four, for the same capability, the electronic device 100 can establish cooperative connections with multiple devices simultaneously, or at most establish cooperative connections with one device; no specific limitation is made here.

[0274] For example, Figures 24A to 24E The user interface for establishing collaborative connections between electronic device 100 and two other devices is shown.

[0275] like Figure 24A , Figure 24B and Figure 24C As shown, the electronic device 100 has established a collaborative connection with the tablet computer's audio output function and camera; the user can attach at least one capability badge corresponding to the collaborative capabilities of the candidate device (e.g., capability badge 503 corresponding to the audio output function) to the device's icon 301. Figure 24C and Figure 24D As shown, the user can drag the device's icon 301 to the area where the laptop's icon 303 is located by sliding; in response to the sliding operation, the electronic device 100 performs a cooperative connection to the laptop's audio output function. Figure 24E As shown, when the collaborative connection to the laptop's camera is successfully confirmed, the display status of the laptop's icon 303 is changed from a non-collaboration state to a collaborative state. An audio output capability badge 503 is added to icon 303, and icon 301 is pushed back into the circular area. (As shown...) Figure 24F As shown, when it is determined that the collaborative connection to the laptop's camera has failed, icon 301 will bounce back to the circular area.

[0276] For details, please refer to Figures 17A to 18C The relevant descriptions will not be repeated here.

[0277] Scenario 5: If the user does not select additional capability badges for candidate device 200 or electronic device 100, electronic device 100 can automatically select the capability for collaborative connection based on the current application scenario.

[0278] In scenarios one and three above, electronic device 100 and candidate device 200 only support one collaborative capability. If the user does not select an additional display capability badge for candidate device 200 or electronic device 100, dragging the icon of candidate device 200 close to the icon of electronic device 100 or dragging the icon of electronic device 100 close to the icon of candidate device, electronic device 100 can automatically select a capability from the collaborative capabilities supported by candidate device 200 for collaborative connection based on the current application scenario.

[0279] In scenarios two and four above, electronic device 100 and candidate device 200 support at least one collaborative capability. If the user does not select an additional display capability badge for candidate device 200 or electronic device 100, dragging the icon of candidate device 200 close to the icon of electronic device 100 or dragging the icon of electronic device 100 close to the icon of candidate device, electronic device 100 can automatically select at least one capability from the collaborative capabilities supported by candidate device 200 for collaborative connection based on the current application scenario.

[0280] In some embodiments, the candidate device 200 in the collaborative connection interface 13 supports at least one collaborative capability. When neither the icon 301 of the candidate device 200 nor the electronic device 100 has a display capability badge attached, the user can drag the icon of the candidate device 200 close to the icon of the electronic device 100 or drag the icon of the electronic device 100 close to the icon of the candidate device by sliding. In response to the above sliding operation, the electronic device 100 automatically selects at least one capability (e.g., capability 1) from the collaborative capabilities of the candidate device 200 for collaborative connection based on one or more of the following: the application 1 that has been running in the foreground most recently, the ranking of the number of times each capability has been collaboratively connected with the candidate device 200, the capability that has been most recently collaboratively connected with the candidate device 200, and the priority of each collaborative capability of the candidate device 200.

[0281] In one implementation, the electronic device 100 first determines the capability (e.g., capability 1) required by the recently running foreground application 1, and automatically selects capability 1 from the collaborative capabilities of the candidate device 200 for collaborative connection. When the collaborative capabilities of the candidate device 200 do not include the capability required by application 1, the electronic device 100 automatically selects at least one capability (e.g., capability 1) from the collaborative capabilities of the candidate device 200 for collaborative connection based on one or more of the following: the ranking of the number of times each capability has been collaboratively connected with the candidate device 200, the capability most recently collaboratively connected with the candidate device 200, and the priority of each collaborative capability of the candidate device 200. In another implementation, the electronic device 100 selects at least one capability (e.g., capability 1) with the highest ranking of the number of times it has been collaboratively connected with the candidate device 200 for collaborative connection; wherein the ranking of the number of times can refer to the ranking of the number of collaborative connections of each capability within the most recent preset time period (e.g., the most recent week), or the ranking of the number of collaborative connections of each capability within the most recent preset total number of connections (e.g., the most recent 10 times). In one implementation, electronic device 100 selects at least one capability that has most recently been cooperatively connected with candidate device 200 for cooperative connection. In another implementation, electronic device 100 selects at least one capability with the highest priority among the cooperative capabilities of candidate device 200 for cooperative connection; wherein, the priority of each capability may be preset by electronic device 100 or preset by user, and is not specifically limited here.

[0282] In some embodiments, the capabilities required by application 1 include capability 1. When application 1 is running in the foreground on electronic device 100, a collaborative connection interface 13 is brought up. When all candidate devices (e.g., candidate device 200) in the collaborative connection interface 13 support capability 1, and neither the icons 301 of candidate device 200 nor electronic device 100 have attached display capability badges, the user can drag the icon of candidate device 200 close to the icon of electronic device 100 or drag the icon of electronic device 100 close to the icon of candidate device 200 by sliding. In response to the above sliding operation, electronic device 100 automatically selects the capability (e.g., capability 1) required by application 1 from the capabilities supported by candidate device 200 for collaborative connection.

[0283] For example, Figures 25A to 25D The user interface shows the ability to determine collaborative connections based on the current application scenario.

[0284] like Figure 25A and Figure 25B As shown, taking a music app as an example, when the music app is running in the foreground on the electronic device 100, it displays the collaborative connection interface 13 in response to the user's input. The required capabilities of the music app include audio output functionality, and all candidate devices displayed in the circular area of ​​the collaborative connection interface 13 support audio output functionality. For example... Figure 25B and Figure 25C As shown, in the collaborative connection interface 13, neither the tablet computer icon 302 nor the electronic device 100 icon 301 has a display capability badge. The user drags the tablet computer icon 301 close to the device's icon 301 by sliding. In response to the sliding operation, the user selects the capability required by the music app (e.g., audio output function) from the tablet computer's collaborative capabilities to establish a collaborative connection. Figure 25D As shown, when the audio output function of the tablet computer is successfully connected, the tablet computer's icon 302 is switched to the connected state, and a capability badge 503 for the audio output function is added to the icon 302.

[0285] In this embodiment of the application, in the aforementioned fifth case, the electronic device 100 can also support establishing collaborative connections with multiple devices simultaneously, which will not be elaborated here.

[0286] The following examples illustrate the application of collaborative connectivity capabilities in the embodiments of this application.

[0287] For example, with Figure 26A The collaborative connection interface 13 shown is used as an example for explanation. Figure 26A As shown, based on the capability labels displayed by the available devices that have been collaboratively connected, the audio output function of the tablet computer and the camera, the audio output function of the laptop computer and the printing function of the printer have been collaboratively connected with the electronic device 100.

[0288] For example, such as Figure 26B and Figure 26C As shown, after the user clicks the toggle control 311 in the music playback interface 14, the electronic device 100 displays a selection box. This selection box includes options for available devices with currently synchronized audio output functionality (i.e., option 801 for tablets and option 802 for laptops), as well as a confirm control 803 and a reject control 804. Figure 26D As shown, after a user selects an option for at least one available device (e.g., option 801 on a laptop), an input operation (e.g., a click operation) is detected on the confirm control 803. In response to this input operation, the electronic device 100 can call the laptop's audio output function to play music. The reject control 804 can receive user input operations (e.g., click operations). In response to this input operation, the electronic device 100 stops displaying the selection box.

[0289] In some embodiments, electronic device 100 can collaboratively connect to the same capability (e.g., capability 1) of multiple devices. In one implementation, for capability 1 of multiple collaboratively available devices, electronic device 100 supports calling capability 1 of only one available device at a time. In another implementation, for capability 1 of multiple collaboratively available devices, electronic device 100 supports simultaneously calling capability 1 of multiple available devices. For example, a user selects... Figure 26D After selecting option 801 for the tablet computer and option 802 for the laptop computer, the electronic device 100 detects an input operation applied to the confirmation control 803; in response to the input operation, the electronic device 100 can simultaneously call the audio output function of the tablet computer and the laptop computer to play music.

[0290] In some embodiments, when the electronic device 100 needs to invoke the capability 1 of an available device, it is not necessary to display the option of the already connected available devices for the user to select. The electronic device 100 can automatically select at least one available device from the connected available devices and invoke the capability 1 of the at least one available device. For example, as shown below... Figure 26B As shown, in response to an input operation to the switching control 311, the electronic device 100 can automatically select an available device with the capability 1 from among available devices with cooperative audio output functionality (i.e., tablet computers and laptops); for example, the electronic device 100 automatically selects an available device with a higher number of cooperative audio output connections, or an available device that has recently made a cooperative audio output connection. No specific limitations are placed on the selection strategy of the electronic device 100 here.

[0291] In some embodiments, if the electronic device 100 only has the capability 1 of one available device connected in collaboration, when the electronic device 100 needs to call the capability 1 of other devices, there is no need to display the option of the available devices that have been connected in collaboration for the user to select. The electronic device 100 can directly call the capability 1 of the available devices that have been connected in collaboration.

[0292] It should be noted that, Figures 26A to 26D This is an exemplary illustration of the application of the capabilities of available devices that have been coordinated in this application, and should not be construed as limiting the scope of this application.

[0293] Referring to the aforementioned collaborative connection interface 13, the collaborative connection interface 13 provided in this application embodiment adopts a two-layer ring layout design. First, the first-layer ring layout reflects the association between other devices and this device; that is, the first-layer ring displays icons of candidate devices detected by the device that can collaborate. The collaboration status (non-collaboration or collaborative status) of the candidate device's icon indicates whether a collaborative connection has been established between this device and the candidate device. Thus, through the first-layer ring layout, the user can intuitively see one or more candidate devices that can currently collaborate with this device, as well as the collaboration status of the candidate devices, facilitating the user's selection of candidate devices to collaborate with. Second, the second-layer ring layout reflects the collaborative capabilities supported by the candidate devices, displaying the capability badges corresponding to the collaborative capabilities. Thus, through the second-layer ring layout, the user can intuitively see the various collaborative capabilities of the candidate devices. The user can select and switch the capability badges displayed alongside the candidate device's icon through the capability badges shown in the second-layer ring, thereby achieving the selection and switching of collaborative connection capabilities. After establishing a collaborative connection with the capability 1 of the candidate device 200, the icon of the candidate device 200 in the collaborative state is additionally displayed with a capability badge, which can also intuitively indicate to the user the collaborative capabilities of the candidate device 200.

[0294] In this embodiment, the two-layer circular layout of the collaborative connection interface 13 features a harmonious and unified page style, resulting in a pleasing visual effect. The user interface interactions—selecting capabilities to be collaborated with, implementing collaborative connections, and canceling collaborative connections—are highly engaging and fun, effectively improving the user experience.

[0295] In some embodiments, the electronic device 100 supports at least one collaborative capability, and the icon displayed on the electronic device 100 is a capability badge of any one of the at least one capability of the electronic device 100 (e.g., capability 1). By sliding the icon of a candidate device close to the icon of the electronic device 100 or by dragging the icon of the electronic device 100 close to the icon of the candidate device, the user can achieve collaborative connection between the candidate device 200 and capability 1 of the electronic device 100; after successful collaborative connection, the candidate device 200 can call capability 1 of the electronic device 100.

[0296] For example, refer to Figure 17A and Figure 17B When a user clicks on icon 301 of electronic device 100, electronic device 100 can display capability badges around icon 301 indicating at least one of its synergistic capabilities; (See reference) Figures 17B to 17DUsers can select one or more capability badges to be displayed on icon 301. Users can drag the icon of candidate device 200 (e.g., icon 302 of a tablet computer) to icon 301 by performing a swipe operation; in response to the swipe operation, electronic device 100 executes the collaborative connection process of candidate device 200 to capability 1 of electronic device 100; after successful collaborative connection, electronic device 100 can switch the display state of candidate device 200 to the collaborative state.

[0297] In one implementation, the electronic device 100 executes a collaborative connection process between the candidate device 200 and the electronic device 100's capability 1, which may include: the electronic device 100 sending a collaborative connection request 5 to the candidate device 200, the collaborative connection request being used to request the candidate device 1 to collaboratively connect to the electronic device 100's capability 1, and the collaborative connection request 5 carrying collaborative connection parameters of the electronic device 100's capability 1; the candidate device 200 establishing a collaborative connection with the electronic device 100's capability 1 based on the aforementioned collaborative connection parameters, and sending a collaborative connection success response to the electronic device 100.

[0298] In some embodiments, if electronic device 100 cooperatively connects to capability 1 of candidate device 200, the icon of candidate device 200 in the cooperative connection interface is displayed as cooperative state 1; when candidate device 200 cooperatively connects to capability 1 of electronic device 100, the icon of candidate device 200 in the cooperative connection interface is displayed as cooperative state 2; cooperative state 1 is different from cooperative state 2. For example, Figure 19D The tablet computer icon 302 is shown in its coordinated state 1. Figure 19C The icon 302 of the tablet computer is shown in its collaborative state 2.

[0299] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100. Of course, in other operating systems, as long as the functions implemented by each functional module are similar to those in the embodiments of this application, the software structure can be adapted accordingly.

[0300] Figure 27 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0301] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the HAL (hardware abstraction layer), and the kernel layer.

[0302] The application layer can include a series of application packages.

[0303] like Figure 27 As shown, apps such as games, camera, navigation, calendar, gallery, Bluetooth, music, SMS, and Word can be installed in the application layer.

[0304] It should be noted that when electronic device 100 runs applications installed in the application layer, it may need to call specific capabilities. For example, a music app needs to call the audio output function when playing music, a camera app needs to call the camera when taking a picture, and a Word app needs to call the print function when printing a Word document.

[0305] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0306] like Figure 27 As shown, the application framework layer may include a window manager, display module, content provider, view system, resource manager, notification manager, etc.

[0307] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0308] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0309] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0310] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0311] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0312] like Figure 27 As shown, in some embodiments, the application framework layer may also include an audio playback module and an image acquisition module.

[0313] For example, the audio player can be a player such as AudioTrack or MediaPlayer. When an audio app needs to play audio, it can invoke the audio player and input the corresponding audio data. For instance, the audio app can input raw audio data into the audio player, which will then process the raw audio data through parsing, de-encapsulation, or decoding to obtain frames of PCM (pulse code modulation) data. Alternatively, the audio app can also directly output PCM data to the audio player. The audio player is used to invoke the speaker to output audio.

[0314] When the camera app needs to capture images, it can invoke the image acquisition module by inputting instructions to it. The image acquisition module is used to call the camera to capture images.

[0315] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0316] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0317] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0318] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0319] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0320] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0321] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0322] A 2D graphics engine is a graphics engine for 2D drawing.

[0323] like Figure 27 As shown, a HAL (Hardware Abstraction Layer) can also be included between the application framework layer and the kernel layer of the Android system. The HAL is responsible for interacting with the various hardware devices of the electronic device 100. On the one hand, the HAL hides the implementation details of each hardware device; on the other hand, it provides the Android system with interfaces for calling these hardware devices. The HAL provides HALs corresponding to different hardware devices, such as the Audio HAL, Camera HAL, and Display HAL.

[0324] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0325] The audio playback module can call the audio HAL to send audio data. The audio HAL then calls the kernel-level audio driver to send the audio data to the device's audio output device (e.g., a speaker) to drive the device's audio output device to play the audio data. Similarly, the image acquisition module can call the camera HAL to send image acquisition instructions. The camera HAL then calls the kernel-level camera driver to drive the device's camera to acquire images.

[0326] In some embodiments, such as Figure 28As shown, a Device Virtualization (DV) App, which is used to implement capability collaboration, can be installed in the application layer of electronic device 100. The DV App can reside permanently in electronic device 100 as a system application. Alternatively, the capability collaboration implemented by the DV App can also reside permanently in electronic device 100 as a system service. When electronic device 100 successfully connects with capability 1 (e.g., audio output function, camera, etc.) of candidate device 200, electronic device 100 obtains the capability parameters of capability 1 of candidate device 200. The DV App can then create a virtual device corresponding to candidate device 200 in electronic device 100 according to these capability parameters. This virtual device possesses capability 1; thus, electronic device 100 can invoke capability 1 of candidate device 200 through this virtual device.

[0327] For example, taking candidate device 200 as a tablet computer, Figure 28 The HAL layer shown also includes the tablet HAL created by the DV app. See also Figures 6A to 6D When electronic device 100 detects a successful collaborative connection between its camera and the tablet computer's camera, the DV APP obtains relevant information about the tablet computer (such as collaborative capabilities, device identifier, device name, device model, and device type) and the camera's capability parameters. It then creates a virtual device call interface corresponding to the tablet computer at the HAL layer (for ease of description, this call interface is simply referred to as the Tablet Computer DV HAL). The Tablet Computer DV HAL includes the corresponding camera HAL for the tablet computer. See also... Figures 14A to 14D After the electronic device 100 successfully connects with the tablet's camera, and then detects that it has also successfully connected with other capabilities of the tablet (such as audio output), the DV APP obtains the capability parameters of the tablet's audio output function (such as playback latency, audio sampling rate, etc.) and creates the corresponding audio HAL for the tablet in the tablet's DV HAL based on these parameters. After the electronic device 100 detects that the user has accessed the tablet's camera, the camera APP sends instruction 2 to the image acquisition module, instructing the use of the tablet's camera to acquire images. The image acquisition module then sends a camera access request to the tablet by accessing the camera HAL in the tablet's DV HAL.

[0328] See Figure 29A collaborative app for enabling capability collaboration can be installed in the application layer of the tablet computer. When the electronic device 100 establishes a collaborative connection with the camera of the tablet computer, it can obtain the camera capability parameters of the tablet computer from the collaborative app. After the collaborative connection is successful, the collaborative app can receive a camera access request sent by the tablet computer; in response to the access request, the collaborative app can send an instruction to the image acquisition module of the tablet computer's application framework layer to instruct the image acquisition module to acquire an image by calling the camera HAL of the tablet computer's HAL layer; the camera HAL can call the camera driver to drive the tablet computer's camera to acquire image 1; the collaborative app acquires image 1 and sends image 1 to the camera app of the electronic device 100; the camera app can display image 1 in the image preview area of ​​the shooting interface 15.

[0329] For example, taking candidate device 200 as a laptop computer YYY, Figure 28 The HAL layer shown also includes the DV HAL for laptops created by the DV app. See also Figures 16A to 16C When electronic device 100 detects a successful collaborative connection between its audio output function and the laptop, the DV APP obtains relevant information about the laptop and the capability parameters of its audio output function. Based on these parameters, it creates a virtual device call interface (referred to as Laptop DV HAL) at the HAL layer. The Laptop DV HAL includes an audio HAL generated based on the aforementioned audio output function capability parameters. After electronic device 100 detects a user input operation to call the laptop's audio output function, the music APP sends instruction 1 and audio data 1 to the audio playback module. Instruction 1 instructs the laptop's audio output function to play audio data 1. The audio playback module, by calling the audio HAL in the Laptop DV HAL, sends an audio output function call request to the laptop, which includes audio data 1.

[0330] See Figure 30A collaborative app can also be installed in the application layer of the laptop. When the electronic device 100 establishes a collaborative connection with the audio output function of the laptop, it can obtain the capability parameters of the audio output function from the collaborative app of the laptop. After the collaborative connection is successful, the collaborative app can receive the audio output function call request sent by the electronic device 100; in response to the above call request, the collaborative app can send an instruction to the audio playback module of the laptop's application framework layer to instruct the audio playback module to play audio data 1 by calling the audio HAL of the laptop's HAL layer; the camera HAL can call the audio driver to drive the tablet's speaker to play audio data 1.

[0331] For example, taking candidate device 200 as printer EEE, Figure 28 The HAL layer shown also includes the printer DV HAL created by the DV APP. When the electronic device 100 detects a successful collaborative connection between the electronic device 100 and the printer's printing function, the DV APP obtains relevant printer information and printing function capability parameters, and creates a virtual device call interface corresponding to the printer in the HAL layer (for ease of description, this call interface is simply referred to as the printer DV HAL); the printer DV HAL includes the print HAL generated based on the aforementioned printing function capability parameters. The application framework layer of the electronic device 100 may also include a print management module. After the electronic device 100 detects the user's input operation to call the printer's print function, the Word APP sends instruction 3 and document data 1 to the print management module. Instruction 3 is used to instruct the printer to call the printer's print function to print document data 1; the print management module sends a print function call request to the printer by calling the print HAL in the printer DV HAL, and this call request includes document data 1.

[0332] See Figure 31 A collaborative app can also be installed in the printer's application layer. When the electronic device 100 establishes a collaborative connection with the printer's printing function, it can obtain the printing function's capability parameters from the printer's collaborative app. After a successful collaborative connection, the collaborative app can receive printing function call requests sent by the electronic device 100; in response to the aforementioned call requests, the collaborative app can send instructions to the printer's application framework layer's print management module to instruct the print management module to print HAL document data 1 by calling the printer's HAL layer.

[0333] Based on the foregoing Figures 1A to 31 The embodiments described herein are illustrated below, along with the device collaboration method provided in this application.

[0334] For example, Figure 32This is a flowchart illustrating a device collaboration method provided in an embodiment of this application. The method is applied to a first electronic device and includes:

[0335] S101, Display a first user interface; the first user interface includes a first icon of a first electronic device and an icon of at least one detected candidate device, the icon of the at least one candidate device includes a second icon of a second electronic device, the second electronic device supports at least two collaborative capabilities, the at least two collaborative capabilities include the first capability.

[0336] The first electronic device can be the electronic device 100 described in the foregoing embodiments, and the second electronic device can be the candidate device 200 (e.g., a tablet computer) described in the foregoing embodiments; the first user interface can be the collaborative connection interface 13 described in the foregoing embodiments, the first icon can be the icon 301 in the collaborative connection interface 13, and the second icon can be the icon 302 in the collaborative connection interface 13.

[0337] For example, the second electronic device is a tablet computer described in the foregoing embodiments, and the tablet computer's collaborative capabilities include support for a camera, audio input functionality, and audio output functionality. The first capability can be any of the aforementioned collaborative capabilities of the tablet computer.

[0338] S102, Receive a first input operation for selecting the first capability.

[0339] S103, Receive the second input operation.

[0340] S104. In response to the second input operation, establish a cooperative connection with the second electronic device that supports the first capability.

[0341] In some embodiments, the first user interface further includes: capability badges corresponding to the collaborative capabilities supported by the second electronic device; the capability badges corresponding to the collaborative capabilities supported by the second electronic device include a first capability badge corresponding to the first capability, and the first input operation acts on the first capability badge. In one implementation, the first user interface displays capability badges corresponding to the collaborative capabilities supported by the second electronic device in the area surrounding the second icon.

[0342] For example, refer to Figure 8B , Figure 8E , Figure 9A and Figure 12B In the collaborative connection interface 13, the area surrounding icon 302 displays capability badges corresponding to the tablet's collaborative capabilities. For example... Figure 8B , Figure 8E and Figure 9AAs shown, the first capability marker can be the capability marker 502 corresponding to the audio input function, and the first input operation for selecting the audio input function is applied to the capability marker 502.

[0343] In some embodiments, prior to receiving the first input operation for selecting a first capability, the method further includes: receiving a third input operation applied to a second icon; and in response to the third input operation, displaying the capability badge corresponding to the collaborative capability supported by the second electronic device in a first user interface.

[0344] For example, refer to Figures 8A to 8G as well as Figures 12A to 12F Related descriptions, such as Figure 8A and Figure 12A As shown, the third input operation may include an input operation (e.g., a click operation) applied to icon 302. It can be understood that after the user clicks the tablet computer's icon 302, the electronic device 100 displays capability badges corresponding to the tablet computer's collaborative capabilities in the collaborative connection interface 13.

[0345] In some embodiments, the method further includes: in response to a first input operation, attaching a first capability badge to a second icon.

[0346] For example, refer to Figures 8A to 8G According to the relevant description, after the user selects capability icon 502, that is, after the electronic device 100 receives the first input operation applied to capability icon 502, capability icon 502 is displayed on icon 302. (Reference) Figures 12A to 12D According to the relevant description, after the user selects capability icon 501 (or capability icon 503), that is, after the electronic device 100 receives the first input operation applied to capability icon 501 (or capability icon 503), capability icon 501 (or capability icon 503) is displayed on icon 302.

[0347] In some embodiments, in response to a third input operation, capability badges corresponding to the collaborative capabilities supported by the second electronic device are displayed in a first area; the at least two collaborative capabilities further include a second capability, and when a third input operation is received on the second icon, a second capability badge corresponding to the second capability is additionally displayed on the second icon; after receiving the third input operation on the second icon, the method further includes: receiving a fourth input operation on the second capability badge in the first area; in response to the fourth input operation, the additional display of the second capability badge on the second icon is stopped.

[0348] For example, refer to Figures 12A to 12F Related descriptions, such as Figure 12A As shown, when a third input operation is received on icon 302, a capability badge 502 is added to icon 302; as Figure 12DAs shown, a fourth input operation (e.g., a touch operation) is received acting on the capability marker 502 in the first area (e.g., the area surrounding icon 302); as Figure 12E As shown, in response to the fourth input operation, the display capability badge 502 is stopped on icon 302.

[0349] In some embodiments, among the capability badges displayed around the second icon, the capability badge with the second icon attached is displayed as selected, and the capability badge without the second icon attached is displayed as unselected; the method further includes: in response to a first input operation, switching the first capability badge displayed around the second icon from an unselected state to a selected state; the method further includes: in response to a fourth input operation, switching the second capability badge displayed around the second icon from a selected state to an unselected state.

[0350] For example, refer to Figures 12A to 12F Related descriptions, such as Figures 12B to 12E As shown, icons 302 with attached displayable capability badges are selected, while icons 302 without attached displayable capability badges are unselected; as... Figure 12B and Figure 12C As shown, in response to the first input operation applied to the capability badge 501, the capability badge 501 displayed around the icon 302 is switched from an unselected state to a selected state; as Figure 12D and Figure 12E As shown, in response to the fourth input operation applied to the capability badge 502, the capability badge 502 displayed around the icon 302 is switched from a selected state to an unselected state.

[0351] In some embodiments, the first input operation includes an operation applied to a first capability icon; the second input operation includes: dragging a second icon toward a first icon until the second icon touches the area where the first icon is located; or, dragging a second icon toward a first icon until the distance between the second icon and the first icon is less than or equal to a preset threshold; or, dragging a first icon toward a second icon until the first icon touches the area where the second icon is located; or, dragging a first icon toward a second icon until the distance between the first icon and the second icon is less than or equal to a preset threshold.

[0352] For example, refer to Figures 6A to 6D According to the relevant description, the second input operation may include: dragging icon 302 towards icon 301 until icon 302 touches the area where icon 301 is located; or, dragging icon 302 towards icon 301 until the distance between icon 302 and icon 301 is less than or equal to a preset threshold. For example, refer to... Figures 10A to 10DAccording to the relevant description, the second input operation may include: dragging icon 301 towards icon 302 until icon 301 touches the area where icon 302 is located; or, dragging icon 301 towards icon 302 until the distance between icon 302 and icon 301 is less than or equal to a preset threshold. For example, the preset threshold is a length of 5 pixels.

[0353] In some embodiments, the second input operation acts on the first capability superscript.

[0354] In some embodiments, the first input operation includes an operation on a first capability icon; the second input operation includes: dragging the first capability icon toward a first icon until the first capability icon touches the area where the first icon is located; or, dragging the first capability icon toward the first icon until the distance between the first capability icon and the first icon is less than or equal to a preset threshold. For example, the first capability icon is capability icon 502.

[0355] For example, refer to Figures 9A to 9C The description states that both the first and second input operations apply to the capability icon 502. For example, the first input operation includes a touch operation on the capability icon 502; the second input operation includes: dragging the capability icon 502 towards the icon 301 until the capability icon 502 contacts the area of ​​the icon 301; or, dragging the capability icon 502 towards the icon 301 until the distance between the capability icon 502 and the icon 301 is less than or equal to a preset threshold. For example, the preset threshold is a length of 5 pixels.

[0356] In some embodiments, the at least two collaborative capabilities further include a second capability; after successfully establishing a collaborative connection supporting the first capability with the second electronic device, the method further includes: receiving a fifth input operation applied to the second icon; in response to the fifth input operation, displaying capability badges corresponding to the collaborative capabilities supported by the second electronic device, wherein the capability badges corresponding to the collaborative capabilities supported by the second electronic device include a first capability badge corresponding to the first capability and a second capability badge corresponding to the second capability; receiving a sixth input operation applied to the second capability badge; and in response to the sixth input operation, establishing a collaborative connection supporting the second capability with the second electronic device.

[0357] For example, refer to Figures 14A to 14E Related descriptions, such as Figure 14A As shown, after establishing a cooperative connection supporting audio input functionality (i.e., the first capability) with the candidate device 200 (e.g., a tablet computer), the electronic device 100 receives a fifth input operation (e.g., a touch operation) applied to the icon 302; as Figure 14BAs shown, in response to the fifth input operation, the capability badges corresponding to the collaborative capabilities supported by the tablet are displayed, such as capability badge 501 corresponding to the camera (i.e., the second capability badge corresponding to the second capability); as Figure 14B As shown, a sixth input operation (e.g., a touch operation) is received acting on capability index 501; as Figure 14C and Figure 14D As shown, in response to the sixth input operation, a collaborative connection with the tablet computer that supports the camera is established.

[0358] In some embodiments, the at least two collaborative capabilities further include a second capability; the step of establishing a collaborative connection with a second electronic device supporting the first capability in response to a second input operation specifically includes: establishing a collaborative connection with a second electronic device supporting both the first and second capabilities in response to a second input operation.

[0359] For example, refer to Figures 13A to 13D According to the relevant description, the tablet computer's collaborative capabilities include camera and audio output functions (i.e., first and second capabilities). In response to a second input operation acting on icon 302 (e.g., dragging from icon 301 to icon 302), the electronic device 100 establishes a collaborative connection with the tablet computer that supports camera and audio output functions.

[0360] In some embodiments, when receiving a second input operation, the second icon is additionally displayed with a first capability icon corresponding to the first capability and a second capability icon corresponding to the second capability.

[0361] For example, refer to Figures 13A to 13D According to the relevant description, when a second input operation is received on icon 302 (e.g., dragged from icon 301 to icon 302), icon 302 will additionally display a capability badge 501 corresponding to the camera and a capability badge 503 corresponding to the audio output function.

[0362] In some embodiments, after successfully establishing a collaborative connection supporting the first capability and the second capability with the second electronic device, the method further includes: receiving a seventh input operation applied to the second icon; in response to the seventh input operation, displaying capability badges corresponding to the collaborative capabilities supported by the second electronic device, wherein the capability badges corresponding to the collaborative capabilities supported by the second electronic device include a first capability badge corresponding to the first capability and a second capability badge corresponding to the second capability; receiving an eighth input operation applied to the first capability badge; and in response to the eighth input operation, canceling the collaborative connection supporting the first capability with the second electronic device.

[0363] For example, refer to Figures 15A to 15D Related descriptions, such as Figure 15AAs shown, after the electronic device 100 establishes a collaborative connection with the tablet computer supporting camera and audio input functions, icon 302 can receive a seventh input operation from the user (such as a touch operation); as Figure 15B As shown, in response to the seventh input operation, the capability icons corresponding to the collaborative capabilities supported by the tablet computer are displayed in the collaborative connection interface 13, including capability icon 501 corresponding to the collaborative camera and capability icon 502 corresponding to the collaborative audio input function; as Figure 15B As shown, the eighth input operation (e.g., a touch operation) acting on capability index 501 is received; as Figure 15C and Figure 15D As shown, in response to the eighth input operation, the collaborative connection with the tablet's supported camera is canceled.

[0364] In some embodiments, in response to a seventh input operation, among the capability badges displayed around the second icon, the capability badges corresponding to the capabilities supported by the cooperative connection established with the second electronic device (i.e., the first capability badge and the second capability badge) are displayed as selected, while the capability badges corresponding to the capabilities not supported by the cooperative connection established with the second electronic device are displayed as unselected.

[0365] For example, refer to Figures 15A to 15D Related descriptions, such as Figure 15B As shown, after the electronic device 100 establishes a collaborative connection with the tablet computer that supports camera and audio input functions, among the capability badges displayed around the collaborative icon 302, the capability badge 501 corresponding to the collaborative camera and the capability badge 502 corresponding to the audio input function are displayed in the selected state, while the capability badge 503 corresponding to the non-collaborative audio output function is displayed in the unselected state.

[0366] In some embodiments, after successfully establishing a cooperative connection supporting both a first capability and a second capability with the second electronic device, the method further includes: receiving a tenth input operation applied to a second icon; and, in response to the tenth input operation, canceling the cooperative connection with the second electronic device for any capability. For example, the tenth input operation is cancellation operation 1 described in the foregoing embodiments.

[0367] For example, refer to Figures 11A to 11D Related descriptions, such as Figure 11B As shown, after establishing a collaborative connection with the tablet computer, the electronic device 100 receives a tenth input operation applied to icon 301 (e.g., an input operation to drag icon 302 away from icon 301); in response to the tenth input operation, the collaborative connection with the tablet computer for any capability is canceled.

[0368] In some embodiments, the first user interface further includes capability labels corresponding to at least two capabilities; the at least two capabilities include all collaborable capabilities supported by the at least one candidate device, the capability labels corresponding to the at least two capabilities include a first capability label corresponding to a first capability, and the first input operation is applied to the first capability label.

[0369] For example, refer to Figure 17B and Figure 19F In the collaborative connection interface 13, the icons around icon 301 are displayed with capability badges corresponding to all collaborative capabilities supported by all candidate devices detected. The first capability badge can be the capability badge 503 corresponding to the audio output function. The first input operation used to select the audio output function is applied to capability badge 503.

[0370] In some embodiments, prior to receiving the first input operation for selecting the first capability, the method further includes: receiving a ninth input operation applied to the first icon; and in response to the ninth input operation, displaying capability badges corresponding to the at least two capabilities in the first user interface. In one implementation, in the first user interface, capability badges corresponding to the collaborative capabilities supported by the second electronic device are displayed in an area surrounding the first icon.

[0371] For example, refer to Figure 17A and Figure 17B According to the relevant description, after receiving a ninth input operation (e.g., a click operation) applied to icon 301, electronic device 100 displays capability badges corresponding to all collaborative capabilities supported by all detected candidate devices around icon 301.

[0372] In some embodiments, the method further includes: in response to a first input operation, attaching and displaying a first capability badge on the first icon. For example, the first capability badge is capability badge 503 corresponding to the audio output function.

[0373] For example, refer to Figures 17A to 17D According to the relevant description, after the user selects capability icon 503, that is, after the electronic device 100 receives the first input operation applied to capability icon 503, capability icon 503 is displayed on icon 301.

[0374] In some embodiments, the method further includes: in response to a first input operation, stopping the display of icons of candidate devices that do not have a first collaborative capability in a first user interface.

[0375] For example, refer to Figure 17B and Figure 17EAccording to the relevant description, after the user selects the capability badge 503 corresponding to the audio output function, the electronic device 100 stops displaying the icons of candidate devices that cannot have the audio output function in the collaborative connection interface 13.

[0376] In some embodiments, the method further includes: when a successful collaborative connection supporting the first capability is established with the second electronic device, the second icon switches from a non-collaborative state to a collaborative state, and the second icon is additionally displayed with a first capability badge corresponding to the first capability; when a failed collaborative connection supporting the first capability is established with the second electronic device, the second icon remains in a non-collaborative state.

[0377] For example, refer to Figures 6A to 6D , Figures 10A to 10F According to the relevant description, when a camera-enabled collaborative connection is successfully established with a tablet computer, icon 302 switches from a non-collaborative state to a collaborative state, and icon 302 displays a capability badge 501 corresponding to the camera. For example, refer to... Figures 18A to 18B , Figures 19A to 19D According to the relevant description, when a collaborative connection supporting audio output function is successfully established with a tablet computer, icon 302 switches from a non-collaborative state to a collaborative state, and icon 302 is additionally displayed with the capability badge 503 corresponding to the audio output function.

[0378] In some embodiments, before receiving a second input operation, a first icon is displayed in a first preset area, and the icon of the at least one candidate device is displayed in a second preset area; the non-collaborating state of the second icon is presented as being displayed in the second preset area in a first state; the collaborating state of the second icon is presented as being attached to the first icon in a second state and displayed in the first preset area, or the collaborating state of the second icon is presented as being displayed in the second preset area in a second state. For example, the first state and the second state refer to different icon colors.

[0379] For example, referring to the relevant descriptions of the foregoing embodiments, such as Figure 3C As shown, before collaborative connection is executed, icon 301 is displayed in the first preset area (e.g., the aforementioned preset area 1), and the icon of the candidate device is displayed in the second preset area (e.g., the aforementioned preset area 2); as Figure 3C As shown, the non-coordinated state of icon 302 is presented as a first state (e.g., a light-colored icon) in the preset area 2; as Figure 6D , Figure 10D , Figure 18B and Figure 19D As shown, the coordinated state of icon 302 is presented as a second state (e.g., a dark icon) attached to icon 301 and displayed in the preset area 1; or, as... Figure 10F and Figure 19CAs shown, the coordinated state of icon 302 is displayed in the preset area 2 as a second state (e.g., a dark icon). This embodiment of the application does not specifically limit the coordinated state, the non-coordinated state, the first state, or the second state.

[0380] In some embodiments, after the first electronic device performs the step of "establishing a cooperative connection with the second electronic device that supports the first capability and the second capability", when a cooperative connection supporting the first capability and the second capability is successfully established with the second electronic device, the second icon switches from a non-cooperative state to a cooperative state, and the second icon is additionally displayed with the first capability badge and the second capability badge; when a cooperative connection supporting the first capability is successfully established with the second electronic device, but the cooperative connection does not support the second capability, the second icon switches from a non-cooperative state to a cooperative state, and the second icon is additionally displayed with the first capability badge, but the second capability badge is not additionally displayed.

[0381] For example, refer to Figures 13A to 13J The relevant description states that the electronic device 100 establishes a collaborative connection with the tablet computer, supporting camera and audio output functions; such as... Figure 13D As shown, when a collaborative connection supporting camera and audio output functions is successfully established with the tablet, icon 302 switches from a non-collaborative state to a collaborative state, and icon 302 additionally displays the capability badge 501 corresponding to the camera and the capability badge 503 corresponding to the audio output function; as Figure 13G As shown, when a camera-enabled collaborative connection is successfully established with a tablet computer, and this collaborative connection does not support audio output, icon 302 switches from a non-collaborative state to a collaborative state. Icon 302 displays a capability badge 501, and does not display a capability badge 503. Figure 13J As shown, when a collaborative connection with a tablet fails to be established, icon 302 remains in a non-collaborative state.

[0382] This application also provides a device collaboration method applied to a first electronic device, the method comprising: displaying a first user interface, the first user interface including a first icon of the first electronic device and icons of at least one detected candidate device, the icons of the at least one candidate device including a second icon of a second electronic device; the second electronic device supporting at least two collaborative capabilities, the at least two collaborative capabilities including a first capability; receiving an eleventh input operation; and in response to the eleventh input operation, establishing a collaborative connection with the second electronic device supporting the first capability based on the collaboration state of a first application recently running in the foreground and / or the second electronic device.

[0383] The first electronic device can be the electronic device 100 described in the foregoing embodiments, and the second electronic device can be the candidate device 200 (e.g., a tablet computer) described in the foregoing embodiments; the first user interface can be the collaborative connection interface 13 described in the foregoing embodiments, the first icon can be the icon 301 in the collaborative connection interface 13, and the second icon can be the icon 302 in the collaborative connection interface 13.

[0384] The first capability can be any of the collaborative capabilities of the aforementioned tablet computers.

[0385] For example, refer to Figures 25A to 25D Related descriptions, such as Figure 25B As shown, the eleventh input operation may include the input operation of dragging icon 302 to icon 301.

[0386] In some embodiments, the cooperative state of the second electronic device includes one or more of the following: the number of cooperative connections of each capability of the first electronic device and the second electronic device, the capability of the first electronic device and the second electronic device that was most recently cooperatively connected, and the priority of each cooperative capability of the second electronic device. This application embodiment does not specifically limit the cooperative state of the second electronic device.

[0387] In some embodiments, establishing a collaborative connection for a first capability of a second electronic device based on the collaborative state of a first application and / or a second electronic device that has recently been running in the foreground includes: establishing a collaborative connection for the first capability of the second electronic device when the collaborative capability of the second electronic device includes the first capability required by the first application.

[0388] For example, refer to Figures 25A to 25D The relevant description states that the first application is a music app, and the tablet's collaborative capabilities include the audio output capabilities required by the music app; the electronic device 100 establishes a collaborative connection with the tablet that supports audio output capabilities.

[0389] In some embodiments, establishing a collaborative connection with a first capability of the second electronic device based on the collaborative state of the most recently running first application and / or the second electronic device includes: establishing a collaborative connection with the first capability that has been collaboratively connected with the second electronic device the most times; or, establishing a collaborative connection with the first capability that has been collaboratively connected with the second electronic device most recently; or, establishing a collaborative connection with the first capability with the highest priority among the collaborative capabilities of the second electronic device.

[0390] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0391] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0392] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0393] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A device coordination method, characterized in that, include: The first electronic device displays a first user interface; the first user interface includes a first icon of the first electronic device, a second icon of the second electronic device, and a projection icon corresponding to the first electronic device, the projection icon being displayed around the first icon; The first electronic device receives a first user operation that drags the screen mirroring icon to the second icon; In response to the first user's operation, the first electronic device and the second electronic device establish a collaborative connection for screen projection; After establishing a collaborative connection for screen mirroring with the second electronic device, the screen mirroring icon is displayed around the second icon to indicate that the first electronic device and the second electronic device have established a collaborative connection for screen mirroring.

2. The method according to claim 1, characterized in that, The method further includes: After establishing a collaborative connection for screen mirroring with the second electronic device, the first electronic device instructs the second electronic device to display the screen mirroring screen of the first electronic device.

3. The method according to claim 1 or 2, characterized in that, After establishing a collaborative connection for screen mirroring with the second electronic device, the screen mirroring icon is not displayed around the first icon.

4. The method according to claim 1, characterized in that, Before the first electronic device displays the first user interface, the method further includes: The first electronic device displays a second user interface; the second user interface includes the first icon of the first electronic device, the third icon of the third electronic device, and the projection icon corresponding to the first electronic device, and the projection icon is displayed around the first icon; After receiving a second user operation to drag the screen mirroring icon to the third icon, a prompt message is displayed, indicating that the collaborative connection for screen mirroring with the third electronic device has failed to be established.

5. The method according to claim 4, characterized in that, When the second user operation is received, the third electronic device has already established a collaborative connection for screen projection with other electronic devices.

6. The method according to claim 4, characterized in that, The method further includes: If the collaborative connection for screen projection with the third electronic device fails to be established, the screen projection icon will be displayed again around the first icon.

7. The method according to claim 1, characterized in that, The first user interface also includes a fourth icon for the fourth electronic device and an audio output icon corresponding to the first electronic device, with the audio output icon displayed around the first icon; The method further includes: Receive a third user action of dragging the audio output icon to the fourth icon; In response to the third user's operation, a collaborative connection for audio output is established with the fourth electronic device, and the audio output icon is displayed around the fourth icon.

8. The method according to claim 7, characterized in that, The method further includes: After establishing a cooperative connection for audio output with the second electronic device, the second electronic device is instructed to play audio from the first electronic device.

9. The method according to any one of claims 1 to 8, characterized in that, Before the first user interface displays the screen mirroring icon corresponding to the first electronic device, the method further includes: Receive a fourth user action applied to the first icon; In response to the fourth user operation, the projection icon is displayed around the first icon on the first electronic device.

10. The method according to any one of claims 1 to 8, characterized in that, The first user interface also includes a fifth icon of the fifth electronic device and at least one other icon corresponding to the first electronic device, wherein the at least one other icon is displayed around the first icon; The at least one other icon includes some or all of the following: camera icon, microphone icon, speaker icon, display icon, printer icon, payment icon, image processing icon, WIFI icon, camera icon, audio output icon, and audio input icon; The method further includes: Receive a fifth user operation to drag a sixth icon to the fifth icon, wherein the sixth icon is any one of the at least one other icon; In response to the fifth user operation, the first electronic device establishes a collaborative connection with the fifth electronic device, and the sixth icon is displayed around the fifth icon. The collaborative connection is used to support the collaborative capabilities associated with the sixth icon.

11. The method according to any one of claims 1 to 8, characterized in that, The first electronic device displays a first user interface, including: In response to a sixth user operation, the first electronic device displays the first user interface; the sixth user is used to operate a first control on the control center interface, or a switching control in the application interface of the first application.

12. The method according to any one of claims 1 to 8, characterized in that, The first electronic device and the second electronic device are electronic devices logged into the same account, or the login accounts of the first electronic device and the second electronic device are in a preset relationship.

13. The method according to any one of claims 1 to 8, characterized in that, The first electronic device and the second electronic device communicate via near-field wireless communication technology.

14. The method according to claim 13, characterized in that, The near-field wireless communication technologies include: Bluetooth, Wi-Fi, UWB, NFC, and ZigBee.

15. An electronic device, characterized in that, The device includes a memory and a processor, the memory and the processor being electrically coupled, the memory being used to store program instructions, and the processor being configured to invoke all or part of the program instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-14.

16. A computer storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-14.

17. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 14.

Images