A device interaction method, electronic device and system

Abstract

The application discloses a device interaction method, an electronic device and a system. The application provides a unified entrance for interaction between a core device and multiple associated devices. When multiple devices initiate connection, the super terminal system provides one or more scene-based service options for a user. Different scene-based service options are entrances of corresponding services in different application scenarios. In addition, different atomized services / function combinations and matching devices can be recommended to the user. By implementing the method provided by the application, the user can quickly select scene-based services and / or atomized services according to individual needs, and through user-selected device combinations, the available scene-based services and / or atomized services under the device combinations are provided, detailed and rich use scenarios are listed for the user, the user is guided to fully explore more application scenarios by using the functions of the super terminal, and the effect of recommending scene-based services according to device combinations is achieved.

Description

[0001] This application is a divisional application, the original application number is 202210270366.5, the original application date is March 18, 2022, and the entire contents of the original application are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal, in particular to a device interaction method, an electronic device and a system. BACKGROUND

[0003] With the upgrading of consumption, the number of smart terminal devices owned by users is increasing. In some use scenarios, users often need two or more terminal devices to work together or in coordination to provide certain services or achieve certain functions, which can also be referred to as cross-device services. For example, a mobile phone and a smart screen work together to realize video projection; a mobile phone shares files or pictures with other devices, etc.

[0004] Generally speaking, the same device manufacturer or the same application manufacturer may set different entries and different triggering methods for different cross-device services. Different device manufacturers or different application manufacturers may also set different entries and different triggering methods for the same cross-device service. For example, a user needs to share pictures on electronic device A to electronic device B. The user can select the pictures to be shared on electronic device A, click the sharing control, and then electronic device A starts searching for surrounding devices and displays a device list. The user selects electronic device B in the device list, and electronic device A can send the selected pictures to electronic device B. For another example, a user needs to project the screen content of electronic device A to electronic device B. The user needs to turn on the wireless function of electronic device A and electronic device B, then pair and establish a connection. After that, the user opens an application related to projection on electronic device A, operates the projection control in the application interface, and then electronic device A sends projection data to electronic device B, and electronic device B displays the screen mirror of electronic device A.

[0005] As can be seen from the above, when multiple devices work together, the user operation is cumbersome, the entry is scattered, the human-computer interaction performance is poor, and the user experience is poor. SUMMARY

[0006] The present application provides a device interaction method, an electronic device and a system, which provides a unified entry for interaction between a core device and multiple associated devices. When multiple devices initiate connection, the super terminal system provides one or more scene-based service options for the user, and different scene-based service options are the entry of the corresponding service in different application scenarios. In addition, different atomized services / functional combinations and matching devices can be recommended to the user.

[0007] The above objects and other objects are achieved by features recited in the independent claims. Further implementation forms are apparent from the dependent claims, the description and the drawings.

[0008] In a first aspect, the embodiments of the present application provide a device interaction method, the method comprising: a first device displaying a first interface, the first interface comprising a first control and a second control, the first control being a device option indicating the first device, and the second control being a device option indicating a second device, the second device being a device discovered by the first device; the first device detecting a first user operation, the first user operation being used for associating the first device and the second device by a user; the first device displaying a second interface, the second interface displaying a service option of a first scene service, the first scene service being a scene service supported by the first device and the second device in association according to the first user operation; the first device detecting a second user operation of selecting the first scene service; and the first device instructing the first scene service selected by the second user operation to run.

[0009] Implementing the method of the first aspect can improve the performance of human-computer interaction. The user can quickly select a scene service and / or an atomic service according to personal needs. By selecting a device combination by the user, the available scene service and / or atomic service under the device combination is provided in a targeted manner. The user is provided with detailed and rich use scenarios, which guides the user to fully explore more application scenarios by using the functions of the super terminal, and achieves the effect of recommending a scene service according to a device combination.

[0010] In combination with the first aspect, in some embodiments, in the first interface, the second controls are distributed on a circumference with the first control as the center, and the first control and the second controls are in a separated state. Figure 5 In the example shown, the first device is a core device such as a mobile phone, and the second device is an associated device such as a smart watch, a smart television, or the like.

[0011] In some embodiments, in combination with the first aspect, the positional relationship between the first control and the second control in the first interface can indicate one or more of the following relationships between the first device and the second device: a connection relationship, a direction relationship, a distance relationship, or a signal strength relationship. Wherein, the first control and the second control are separated, indicating that the first device and the second device are not connected, the first control and the second control are connected, indicating that the first device and the second device are connected. And / or, the second control is located on the left, or right, or top, or bottom of the first control, respectively indicating that the second device is located on the left, or right, or front, or back of the first device. And / or, the farther the distance between the first control and the second control, the farther the distance between the first device and the second device, or the weaker the signal between the first device and the second device, the closer the distance between the first control and the second control, indicating that the closer the distance between the first device and the second device, or the stronger the signal between the first device and the second device.

[0012] The device connection state diagram in the first interface can visually and concisely indicate each device and show the relationship between the core device and the associated device, making it easy for the user to operate and understand the interface, and making the user interface design more humanized and the user experience better.

[0013] In some embodiments, in combination with the first aspect, the first user operation includes an operation of the second control being selected, or the first control being selected, or the first control and the second control being selected at the same time.

[0014] In some embodiments, in combination with the first aspect, the first user operation is an operation of the user selecting the second control and dragging the second control to a specified area in a direction close to the first control, and then releasing the second control. Or the first user operation is an operation of the user selecting the first control and the second control at the same time, and dragging the first control and the second control close to each other until the distance between the first control and the second control is less than a certain distance, and then releasing the first control and the second control. The specified area can be a circular area with the first control as the center and a first radius distance as the radius. Or the first user operation is an operation of the user selecting the first control and dragging the first control in a direction close to the second control until the distance between the first control and the second control is less than a certain distance, and then releasing the first control.

[0015] The second interface also displays that the first control and the second control are attracted together, indicating that the first device and the second device have established a connection and can work together to provide system-level services. Reference Figure 6 、 Figure 7The drag operation is more in line with the user's operation habits, and the user has low learning cost and easy operation. The two device options being attracted together can mean that the edges of the icons of the two devices are tangent, or the two device options being attracted together can also mean that the two device icons are completely or partially overlapped. The embodiment does not specially limit the form of the attraction.

[0016] In some embodiments, the second control in the second interface can be different from the form of the second control in the first interface. For example, in Figure 7 , the smart watch icon 702 connected with the mobile phone to form a super terminal can display a distinguished mark, so as to be distinguished from the icon of the associated device in the unconnected state. For example, the smart watch icon 702 is displayed in a dark fill form, indicating that the smart watch is in a connected state with the mobile phone. In addition, the size of the smart watch icon 702 is smaller than that of the smart watch icon 506 in the unconnected state.

[0017] In combination with the first aspect, in some embodiments, the first user operation is an operation of the user clicking the second control or simultaneously clicking the first control and the second control.

[0018] In combination with the first aspect, in some embodiments, the second interface displays a floating control, and the floating control includes a first area displaying service options of the first scene-based service. The first user operation can be an operation of the user holding the second control. The second user operation can be an operation of the user releasing the second control after dragging the second control to the first area, or the second user operation can be an operation of the user releasing the second control after dragging the second control to a specified area in a direction close to the first control via the first area. The specified area can be a circular area with the first control as the center and a first radius distance as the radius. Reference is made to Figure 11 、 Figure 12 、 Figure 13A 、 Figure 13B 、 Figure 13C 、 Figure 13D The described embodiments. In this embodiment, one second user operation includes the functions of associating the second device to the first device and selecting the first scene-based service, thereby improving the user operation efficiency.

[0019] In combination with the first aspect, in some embodiments, the selected device option can change positions along the track of the user's finger dragging on the screen. The embodiment does not limit the track of the finger movement when the user drags the device icon, and the user's dragging track can be any curve.

[0020] In combination with the first aspect, in some embodiments, the floating control includes a plurality of circular areas, or a plurality of rectangular areas, or a plurality of sector areas, or a plurality of sector ring areas, or a plurality of circular ring areas, or a plurality of polygon areas, etc.

[0021] In some embodiments, the first control is included in the second interface. In some embodiments, the first control is not included in the second interface.

[0022] A scenario service refers to a service based on a usage scenario, and is a collection of multiple functions or services that can be provided for a specific scenario. One scenario service can be supported by one or more atomic services. The service options of the scenario service displayed on the second interface can be scenario services that can be provided by the current super terminal system, i.e., the device combination of the first device and the second device, and the atomic capabilities corresponding to these scenario services have been installed and deployed on the first device or the second device.

[0023] In some embodiments, a scenario service can be based on a same application, and different devices provide different atomic capabilities under the application service. The scenario service is actually provided by the same service provider, for example, Figure 8 In the example shown, the mobile phone and the smart watch both install a sports health application. After the user selects the scenario service option of outdoor running, the mobile phone and the smart watch both jump to the outdoor running service in the sports health application. The smart watch can provide a positioning service and a service of counting user sports data, and can upload user data to an application server. The application server synchronously sends the user data to the mobile phone. The mobile phone can provide a service of displaying the user's sports track. For the same scenario service, even if the same client is used, the interfaces displayed by different devices and the services provided can be different.

[0024] In some embodiments, the first scenario service is a scenario service queried from a first database. The first database records one or more scenario services supported by one or more device combinations, and the device combination includes a device combination of the first device and the second device associated therewith. The scenario service can be determined by one or more of the following parameters: device type, device characteristics, product positioning, device purpose, environment or scenario in which the device is located, state in which the device is located, and recently run application of the device combination. Refer to Table 1 in subsequent embodiments. The first database can be stored locally or on a cloud server.

[0025] In some embodiments, the first scenario service is queried from a second database that records one or more scenario services supported by one or more atomic service combinations, the atomic service combination including atomic capabilities possessed by the first device and the second device associated therewith. The atomic capabilities can include one or more of audio output capability, audio input capability, display capability, camera capability, touch input capability, and keyboard / mouse input capability. The second database can be stored locally or on a cloud server.

[0026] For example, the audio output capability can include support for single-channel or multi-channel, supported sound effects, supportable frequency response range, noise reduction capability, and audio resolution capability when the device plays audio. The audio input capability can include sound collection range and noise reduction capability. The display capability can include screen size, display resolution parameter, refresh rate, and color performance of the device. The camera capability can include camera type, shooting pixels, night scene shooting capability, and image adjustment capability of the device. The touch input capability and the keyboard / mouse input capability can indicate whether the device supports touch input or keyboard / mouse input.

[0027] In some embodiments, the first scenario service can be selected according to one or more of the following parameters: frequency of use of the first scenario service by the user, ranking of the first scenario service set by the user, environment or scenario in which the first device and / or the second device is located, state of the first device and / or the second device, and application recently run by the first device and / or the second device. The service option of the first scenario service with a higher probability of being selected by the user is displayed with higher priority.

[0028] For example, different device types have different device characteristics. For example, a smart TV has a large screen and provides a good visual experience for the user, and is more suitable for video watching. A smart speaker has strong audio output capability and good sound effect, and is more suitable for audio playing. An earphone is small and portable, and does not need to be played externally, does not disturb people around, and can have noise reduction effect to shield external environmental noise, and is suitable for listening to audio in public places, such as making a call or listening to music in public places. A smart watch or a smart bracelet can more accurately measure motion data or heart rate and other health information of the user than a mobile phone. A notebook computer has strong processing capability and is more suitable for office work. Here, the device types and device characteristics are not described in detail, and the above examples do not limit other embodiments.

[0029] In some embodiments, when listing the scenario-based service options, the high-usage options can be displayed in the front according to the statistical frequency of user usage of each scenario-based service option in a certain time. The certain time can be all historical time or a certain time period, such as the last three months. The higher the historical usage rate of the scenario-based service option, the more in the front it is displayed.

[0030] In other embodiments, the first device or the second device can report the detected environment where the current device is located, and sort the scenario-based service options according to the environment. If it is detected that the user is located in an outdoor environment, the outdoor-related service options are preferentially recommended, such as outdoor running. If it is detected that the user is located in an indoor environment, the indoor-related service options are preferentially recommended, such as indoor running. The embodiments do not limit the way of detecting the environment where the user is located. For example, the current positioning information of the device can be used to determine the environment; or a camera is used to collect a photo of the current environment, or an ultrasonic wave, infrared, etc. is used to collect the depth or size of the current space to determine the environment where the user is currently located.

[0031] In other embodiments, the first device or the second device can also determine the environment or scenario where the device is located according to the application that the user recently opened or used, or the currently running application, and then recommend the scenario-based service options. For example, in the case of a super terminal composed of a mobile phone and a smart TV, if it is detected that the currently running application of the mobile phone is an office demonstration application, the multi-screen collaboration service option is preferentially recommended; or if it is detected that the currently running application of the mobile phone is a video application, the home theater service option is preferentially recommended; or if it is detected that the user browses a game-related webpage, the game entertainment-related service option is preferentially recommended, and the like.

[0032] In other embodiments, the scenario-based service can also be recommended according to the running state of the device, such as the screen-on or screen-off state. For example, when the mobile phone and the smart TV are connected to form a super terminal, if it is detected that the smart TV is in a screen-on state, the scenario-based service is preferentially recommended as a screen mirroring, and if it is detected that the smart TV is in a screen-off state, the scenario-based service is preferentially recommended as a voice casting service.

[0033] In combination with the first aspect, in some embodiments, in the present embodiment, the layout of the floating control is different, and the arrangement order of the recommended scenario-based services is also different. The developer can design the area layout of the floating control according to the user experience.

[0034] Optionally, as shown in the embodiments of FIGS. 8A and 8B, the shape of the floating control is a circular ring or a fan ring, and the plurality of circular ring areas or fan ring areas respectively indicate a plurality of first scenario-based services. The circular ring area or fan ring area closer to the center indicates a more preferentially recommended scenario-based service. Figure 13C 、 Figure 13D ​

[0035] Optional, see reference Figure 11 , Figure 12 , Figure 13A , Figure 13B In the illustrated embodiment, the floating control is circular or rectangular in shape, and the circle or rectangle is divided into four regions: the first quadrant is region A, the second quadrant is region B, the third quadrant is region C, and the fourth quadrant is region D. The recommended order of the contextual service options is contextual service option A, contextual service option B, contextual service option C, and contextual service option D. The regions where these options are placed are region B, region A, region C, and region D, or region B, region C, region A, and region D, or region A, region D, region B, and region C, or region A, region B, region D, and region C, or region B, region A, region D, and region C, or region A, region B, region C, and region D.

[0036] In conjunction with the first aspect, in some embodiments, the second interface also displays a third device option. This third device is a recommended device based on an assessment of the usage needs of the first device and its associated second device. This helps users expand their usage scenarios and better experience the super terminal ecosystem.

[0037] In conjunction with the first aspect, in some embodiments, if the third device is a device discovered by the first device, clicking on the device option of the third device will display the service options of the second scenario-based service on the first device. The service options of the second scenario-based service may be additional scenario-based service options added on top of the service options of the first scenario-based service, or the service options of the second scenario-based service may be a newly updated scenario-based service, different from the service options of the first scenario-based service. This makes the user operation convenient.

[0038] In conjunction with the first aspect, in some embodiments, if the third device is not a device discovered by the first device, clicking on the device option of the third device can redirect the first device to display a fifth interface. For example, the fifth interface could be a purchase interface for the third device, including multiple item options from multiple providers, including an item option indicating the third device. The fifth interface could include a functional introduction, usage instructions, and suggested usage scenarios for the third device, thus providing users with more convenient services.

[0039] In some embodiments of the first aspect, the third device is recommended based on one or more parameters, including a device type, a device feature, a product positioning, a device usage, an environment or a scenario in which the device is located, a state in which the device is located, a recently run application, or a first scenario service, or a predicted expandable usage scenario, of the first device and / or the associated second device.

[0040] After the recommended device is added, the system can further recommend a new device based on the newly formed super terminal. The user can select to add more devices to form a super terminal, so that the user's usage scenarios are more diverse. After more associated devices are added, the super terminal can provide more and richer scenario services, and perfect the super terminal ecosystem.

[0041] In some embodiments of the first aspect, the first scenario service is supported by a first atomic service combination, which includes at least one first service provided by the first device and / or at least one second service provided by the second device. An atomic service refers to a minimum unit of capability that can be independently run, and is a concept of abstract encapsulation of a single function / capability, which can be a hardware service or a software service.

[0042] In some embodiments of the first aspect, the first device can display the first atomic service combination on a seventh interface, as shown in the embodiment of FIG. 6. Figure 10 The first atomic service combination can be selected by the user, or the first atomic service combination is determined according to one or more parameters, including a device type, a device feature of the first device and / or the second device, or a commonly used atomic service combination of the user, or a default atomic service combination set by the user, or an atomic service combination that is most suitable for the first scenario service obtained by analysis, or an atomic service combination with the highest usage frequency under the first scenario service obtained from a server.

[0043] In some embodiments of the first aspect, the first device runs the first scenario service, and / or the second device associated with the first device runs the first scenario service.

[0044] In some embodiments of the first aspect, the third interface displayed by the first device when running the first scenario service is different from a fourth interface displayed by the second device when running the first scenario service.

[0045] In some embodiments of the first aspect, if the first scenario service is a screen projection service and the second device is a large-screen device, the fourth interface can be a video or image screen, and the third interface can be a control interface for controlling the display function of the video or image screen. For example, as shown in the embodiment of FIG. 7, the fourth interface is a video screen, and the third interface is a control interface for controlling the display function of the video screen. Figure 18The third interface and the fourth interface can display the same screen content, as shown in the embodiments described above with reference to the first aspect. Figure 19 In the embodiments shown in the embodiments described above with reference to the first aspect, the mobile phone and the television display the same video screen.

[0046] In combination with the first aspect, in some embodiments, if the first scene-based service is a sports monitoring service and the second device is a wearable sports device, the third interface can be a screen of a sports track, and the fourth interface can be sports data of the user detected, including sports speed, sports distance, sports time, and heart rate. Figure 8 In the embodiments shown in the embodiments described above with reference to the first aspect, the mobile phone and the television display the same video screen.

[0047] The effect produced and the service triggered can be different when the user drags the second control to the first control from different directions and different areas.

[0048] In combination with the first aspect, in some embodiments, in the screen projection scene, the first device determines the position of the screen interface of the first device in the screen of the second device according to the relative position of the second control and the first control when the drag operation is released.

[0049] In combination with the first aspect, in some embodiments, in the screen projection scene, the first device determines the position of the screen interface of the first device in the screen of the second device according to the relative position of the second control and the first control when the drag operation is released. Figure 15 、 Figure 16 、 Figure 17 In the embodiments described above with reference to the first aspect. If the relative position is that the second control is on the left side of the first control, the screen interface of the first device is projected on the right side of the screen of the second device. If the relative position is that the second control is on the right side of the first control, the screen interface of the first device is projected on the left side of the screen of the second device. If the relative position is that the second control is in the middle of the first control, the screen interface of the first device is projected on the screen of the second device in full screen.

[0050] In another example, for audio devices such as speakers, earphones, etc., different channels can be associated with different orientations of adhesion. Assuming that a user has two or more speakers, the user drags a first speaker icon from the left area of the phone icon to adhere to it, and the first speaker can provide left channel audio output; the user drags a second speaker icon from the right area of the phone icon to adhere to it, and the second speaker can provide right channel audio output. Other multi-channel arrangements are similar.

[0051] In other embodiments, the first control and the second control interchange the dragging direction, and the service or interactive function executed thereby can be different. For example, the data transmission direction is different, and the device as the adhesion side or the adhered side indicates that the device is the data sender or receiver, such as a data transmission direction that the adhesion side device sends data stream to the adhered side device. For example, when sharing a file, if the second control is held and dragged to adhere to the first control, the triggered service is to send the file of the second device to the first device, and if the first control is held and dragged to adhere to the second control, the triggered service is to send the file of the first device to the second device.

[0052] In other embodiments, the device options as the adhesion side or the adhered side can also be different in the triggered scenario service. For example, if the second control is held and dragged to adhere to the first control, the triggered service is to send the file of the second device to the first device, and if the first control is held and dragged to adhere to the second control, the triggered service is to project the screen of the first device onto the screen of the second device.

[0053] In combination with the first aspect, in some embodiments, the method can further include that the first device detects a third user operation acting on the second interface, the third user operation being used to associate a fourth device with the first device, the third user operation including a third control operation of selecting a third control indicating the fourth device. In response to the third user operation, the first device displays a sixth interface, the first control, the second control, and the third control being displayed adhered together in the sixth interface, and the sixth interface further displays a third scenario service option, the third scenario service option being determined according to the first device, the second device, and the fourth device, and the third scenario service option being different from the first scenario service option. Reference is made to Figure 9 the embodiments shown.

[0054] In combination with the first aspect, in some embodiments, after the first device runs the first scenario service selected by the second user operation, the first device detects a fourth user operation, the fourth user operation including an operation of separating the first control from the second control. In response to the fourth user operation, the first device instructs the first scenario service selected by the second user operation to stop running.

[0055] With reference to the first aspect, in some embodiments, in response to the fourth user operation, the first device stops running the first scene-based service selected by the second user operation, and / or the second device stops running the first scene-based service selected by the second user operation.

[0056] With reference to the first aspect, in some embodiments, the fourth user operation can be that the user selects the second control and drags the second control away from the first control until the second control moves out of the specified area and is released. The second control returns to the position before being connected, the attraction state is released, the second device is disconnected from the first device, and the atomized service provided by the first device or the second device is terminated. Alternatively, when the user clicks the second control in the connected state or the user holds the second control and the first control and drags them apart in the opposite direction, the first device and the second device can also be disconnected. On the interface, the second control and the first control are separated, and the second control returns to the position in the unconnected state.

[0057] In some embodiments, a one-key disconnection control can also be provided on the interface of the first device. Clicking the one-key disconnection control can disconnect all connected associated devices from the first device, restore the first device to a state of being independent and having no associated devices connected, and terminate the services provided by the associated devices.

[0058] With reference to the first aspect, in some embodiments, the first device includes any one of a mobile phone, a tablet computer, a portable / non-portable computer, a personal computer, a smart television, etc. The second device includes any one of a mobile phone, a tablet computer, a portable computer, a desktop personal computer, a smart speaker, a smart watch, a smart bracelet, a smart television, a headset, smart glasses, a car machine, a smart cockpit, a game console, a treadmill, a spinning bike, a body scale, a body fat scale, a water heater, a lamp, an air conditioner, a blood glucose meter, a blood oxygen saturation meter, a heart rate meter, an AR / VR device, a cloud host / cloud server, a smart wearable device, a smart home device, a printer, etc.

[0059] With reference to the first aspect, in some embodiments, the first scene-based service can be a printing service, and the second device can be a printer. When the first device and the second device cooperate to run the first scene-based service, the first device can display a printing floating control. The printing floating control can be displayed on the user interface of the first device.

[0060] That is, during the period in which the first device runs the first scene service, the first device can always display the print floating control on the user interface. The content displayed on the user interface of the first device changes, but the print floating control can still be displayed on the user interface. In this way, the user can quickly print the content displayed on the current user interface through the print floating control.

[0061] In combination with the first aspect, in some embodiments, the first device detects a fifth user operation on the print floating control. The first device can send the first print file to the second device for printing based on the fifth user operation, the first print file being the content displayed on the first device when the first device detects the fifth user operation.

[0062] In some embodiments, when the content displayed on the user interface of the first device changes from the first print file to a second print file, the first device can send the second print file to the second device for printing in response to the operation on the print floating control.

[0063] In combination with the first aspect, in some embodiments, the first device detects a sixth user operation on the print floating control. The first device can delete the print floating control and instruct the first scene service to stop running based on the sixth user operation.

[0064] As can be seen, the user can directly trigger the first device and the second device to stop running the first scene service through the print floating control, so that the user does not need to return to the first interface to trigger the first device and the second device to stop running the first scene service through the first control and the second control.

[0065] The user operations of the first device in cooperation with the printer to provide the print service and the user operations to stop the cooperation of the first device and the printer are simple, which can improve the user experience.

[0066] In a second aspect, the embodiments of the present application provide a device interaction method. A first device displays a first interface, the first interface including a first control and a second control, the first control being a device option indicating the first device, and the second control being a device option indicating a second device, the second device being a device discovered by the first device. The first device detects a first user operation, the first user operation being used to associate the first device and the second device. The first device sends a first binding request to the second device according to the first user operation, and receives a first consent message of the second device. The first device displays a second interface according to the first consent message, the second interface displaying a service option of a first scene service, the first scene service being a scene service supported by the first device and the second device in cooperation.

[0067] The method of the second aspect can improve the human-computer interaction performance, the user can quickly select a scenario service and / or an atomic service according to personal needs, the user-selected device combination is targeted to provide the scenario service and / or the atomic service available under the device combination, the user is listed with detailed and rich use scenarios, the user is guided to use the super terminal to fully explore more application scenarios, and the effect of recommending a scenario service according to a device combination is achieved.

[0068] In combination with the second aspect, in some embodiments, the first device can display a connection code setting box according to the first user operation, the connection code setting box being used for setting a connection code for binding the first device and the second device. The first device receives the first connection code input in the connection code setting box and sends a first binding request to the second device. The first consent message contains the first connection code.

[0069] In some embodiments, after the second device receives the first binding request, the second device can ask the user whether to agree to the cooperation of the second device with the first device. If it is detected that the user agrees to the cooperation of the second device with the first device, the second device can display a connection code input box to let the user input a connection code. When receiving the connection code input by the user in the connection code input box, the second device can send the first consent message. After receiving the first consent message, the first device can detect whether the connection code contained in the first consent message is the connection code set in the connection code setting box. If yes, the first device and the second device can complete device binding and establish cooperation, thereby providing the user with a scenario service.

[0070] As can be seen from the above embodiments, the first device and the second device can perform device binding before running the first scenario service. In the above device binding process, the first device can set a connection code, and the second device can input the connection code for device binding. The above device binding process can ask the user to which the second device belongs whether to agree to the cooperation of the second device with the first device. The above embodiments can reduce the probability that the electronic device of the user to which the second device belongs is used by other users when the user to which the second device belongs does not agree or is unaware in the case that the user to which the first device belongs is different from the user to which the second device belongs. Moreover, the verification process of the connection code can ensure the accuracy of the electronic device bound with the first device and improve the user experience.

[0071] In combination with the second aspect, in some embodiments, in the first interface, the second controls are distributed on a circumference with the first control as the center and a first length as the radius, and the first control and the second controls are in a separated state.

[0072] The first user operation can be that the user selects the second control, drags the second control to the specified area in a direction close to the first control, and releases the second control. Alternatively, the first user operation can be that the user selects the first control and the second control at the same time, drags the first control and the second control close to each other until the distance between the first control and the second control is less than a certain distance, and then releases the first control and the second control. The specified area can be a circular area with the first control as the center and a first radius distance. Alternatively, the first user operation can be that the user selects the first control, drags the first control to move in a direction close to the second control, and releases the first control until the distance between the first control and the second control is less than a certain distance.

[0073] In some embodiments, the first control and the second control are also displayed in the second interface. In the second interface, the first control and the second control can be adsorbed together, indicating that the first device and the second device have established a connection and can work together to provide a system-level service. The two device options being adsorbed together can mean that the edges of the icons corresponding to the two devices are tangent, or the two device options being adsorbed together can also mean that the two device icons are completely or partially overlapped.

[0074] The first operation described above conforms to the user's operation habits, has low user learning cost, and is easy to operate.

[0075] In combination with the second aspect, in some embodiments, the first interface further includes a fourth control, the fourth control being a device option indicating a fifth device, the fifth device being a device discovered by the first device, and an account associated with the fifth device being the same as an account associated with the first device; the fourth control is distributed on a circumference with the first control as the center and a second length as the radius, the first control and the fourth control being in a separated state, and the first length being different from the second length.

[0076] In combination with the second aspect, in some embodiments, the account associated with the first device is different from an account associated with the second device, or the second device has no associated account.

[0077] As known from the above embodiments, the device options indicating the cross-account devices (or the devices without an account) and the device options indicating the same-account devices can be displayed on circumferences with the first control as the center and different radii. For example, the device options indicating the same-account devices can be displayed on one circumference. The device options indicating the cross-account devices and / or the devices without an account can be displayed on another circumference. In this way, the user can quickly distinguish which devices are his own and which devices are of other users.

[0078] The same-account device can refer to a device associated with the same account as the first device. The cross-account device can refer to a device associated with a different account than the first device. The no-account device can refer to a device without an associated account.

[0079] In some embodiments, when the second device is a cross-account device or a no-account device, the first device can perform device binding with the second device in response to the first operation. When the first device and the second device perform device binding for the first time, the first device and the second device can perform device binding through the connection code. For example, the first device can display the connection code setting box, receive the connection code set by the user, and send a binding request to the second device. The second device receiving the binding request can ask the user whether to agree to the binding of the second device and the first device. When detecting an operation of agreeing to the binding of the second device and the first device, the second device can display a connection code input box and ask the user to input the connection code for the binding of the second device and the first device. When receiving the connection code input in the connection code input box, the second device can send an agreement message to the first device. The agreement message further includes the connection code received in the connection code input box. When the first device and the second device perform device binding for the second time or the third time, the user operation can be simpler than the user operation when the first device and the second device perform device binding for the first time. For example, when the first device and the second device have performed device binding, the first device can send a binding request to the second device in response to the first user operation. When receiving the binding request, the second device can ask the user whether to agree to the binding of the second device and the first device. When detecting an operation of agreeing to the binding of the second device and the first device, the second device can send an agreement message to the first device.

[0080] In some embodiments, when the first user operation associated with the first device and the second device is received for the first time, the first device and the second device can perform device binding. After the first device and the second device have performed device binding, when the first device receives the user operation associated with the first device and the second device again, the first device can directly establish a connection with the second device and provide a scenario-based service in cooperation.

[0081] In combination with the second aspect, in some embodiments, the first interface further includes a fifth control, and the fifth control is a device option indicating a sixth device discovered by the first device. The fifth control and the fourth control are in a separate state. The first device detects a seventh user operation for associating the fifth device and the sixth device. The first device indicates, according to the seventh user operation, the fifth device to request binding with the sixth device, and receives a second agreement message indicating that the sixth device agrees to bind with the fifth device. The first device displays an eighth interface according to the second agreement message, and in the eighth interface, the fourth control and the fifth control are attracted together.

[0082] The fourth control and the fifth control being adsorbed together can indicate that the first device and the second device have established a connection and can work cooperatively to provide a system-level service.

[0083] As can be seen from the above embodiments, the core device in the super terminal can also associate other same-account devices and cross-account devices in the super terminal in response to a user operation, so that the same-account devices and the cross-account devices can work cooperatively to provide a scenario-based service. Before the same-account devices and the cross-account devices work cooperatively, the devices can be bound.

[0084] In combination with the second aspect, in some embodiments, the first device detects a user operation acting on the second control and removes the second control from the first interface.

[0085] In combination with the second aspect, in some embodiments, the first interface further displays a recycle bin control. The first device detects a user operation acting on the recycle bin control and displays a recycle bin list. The recycle bin list can include one or more device options. The device indicated by the device option in the recycle bin list can be the device indicated by the device option removed from the first interface. Illustratively, when the user removes the second control from the first interface, the user can view the recycle bin list through the recycle bin control. The recycle bin list can include a device option indicating the second device. In response to an operation on the device option indicating the second device in the recycle bin list, the first device can display the second control on the first interface again.

[0086] In combination with the second aspect, in some embodiments, the second control in the second interface can be different from the second control in the first interface in form.

[0087] In combination with the second aspect, in some embodiments, the second interface displays a floating control including a first area displaying a service option of the first scenario-based service. The first user operation can be an operation in which the user holds the second control. The second user operation can be an operation in which the user drags the second control to the first area and releases the second control, or an operation in which the user drags the second control to pass through the first area and moves to a specified area close to the first control and releases the second control. The specified area can be a circular area with the first control as the center and a first radius distance. Figure 11 、 Figure 12 、 Figure 13A 、 Figure 13B 、 Figure 13C 、 Figure 13D The described embodiments. In this embodiment, one second user operation includes the functions of associating the second device to the first device and selecting the first scenario-based service, thereby improving user operation efficiency.

[0088] In some embodiments of the second aspect, the floating control includes a plurality of circular regions, or a plurality of rectangular regions, or a plurality of sector regions, or a plurality of sector ring regions, or a plurality of circular ring regions, or a plurality of polygonal regions, etc.

[0089] In some embodiments of the second aspect, the first control is included in the second interface. The first control can also not be included in the second interface.

[0090] In some embodiments of the second aspect, after the first device receives the first consent message of the second device, the first device can cooperate with the second device to run a first scenario service. The first scenario service can be a scenario service with the highest probability of being selected by the user among K1 scenario services. The K1 scenario services can be scenario services that can be provided by the cooperation of the first device and the second device. K1 is a positive integer. The probability of a scenario service being selected by the user among the K1 scenario services can be calculated according to one or more of the following parameters: the frequency of the user using the scenario service, the ranking of the scenario service set by the user, the environment or scenario in which the first device and / or the second device is located, the state of the first device and / or the second device, and the application recently run by the first device and / or the second device. The K1 scenario services further include a fourth scenario service. The second interface can display a service option of the fourth scenario service. The user can use the service option of the first scenario service to adjust the scenario service provided by the cooperation of the first device and the second device from the first scenario service to the fourth scenario service.

[0091] As can be seen from the above embodiments, the first device can cooperate with the second device to directly provide a scenario service with the highest probability of being selected by the user according to the user operation associated with the first device and the second device. If the scenario service required by the user is not the scenario service with the highest probability of being selected by the user determined by the first device and the second device, the user can select the scenario service he wants through the service option of the other scenario service.

[0092] In a third aspect, the embodiments of the present application provide an electronic device, which can include a communication device, a display device, a memory, and a processor coupled to the memory, a plurality of application programs, and one or more programs. The communication device is configured to communicate, the display device is configured to display an interface, the memory stores computer executable instructions, and the processor executes the instructions to enable the electronic device to implement any function of the first device in the first aspect.

[0093] In a fourth aspect, an electronic device is provided, which can include a communication device, a display device, a memory, and a processor coupled to the memory, a plurality of application programs, and one or more programs. The communication device is configured to communicate, the display device is configured to display an interface, the memory stores computer executable instructions, and the processor executes the instructions to cause the electronic device to implement any of the functions of the first device in the second aspect.

[0094] In a fifth aspect, a computer storage medium is provided, which stores a computer program including executable instructions, and the executable instructions, when executed by a processor, cause the processor to perform operations corresponding to the method in the first aspect.

[0095] In a sixth aspect, a computer storage medium is provided, which stores a computer program including executable instructions, and the executable instructions, when executed by a processor, cause the processor to perform operations corresponding to the method in the second aspect.

[0096] In a seventh aspect, a computer program product is provided, which, when running on an electronic device, causes the electronic device to perform any of the possible implementation manners of the first aspect.

[0097] In an eighth aspect, a computer program product is provided, which, when running on an electronic device, causes the electronic device to perform any of the possible implementation manners of the second aspect.

[0098] In a ninth aspect, a chip system is provided, which can be applied to an electronic device, and the chip includes one or more processors configured to invoke computer instructions to cause the electronic device to implement any of the possible implementation manners of the first aspect.

[0099] In a tenth aspect, a chip system is provided, which can be applied to an electronic device, and the chip includes one or more processors configured to invoke computer instructions to cause the electronic device to implement any of the possible implementation manners of the second aspect.

[0100] In an eleventh aspect, a communication system is provided, which includes a first device and a second device, and the first device implements any of the possible implementation manners of the first aspect.

[0101] By implementing the above aspects provided in the present application, a user can quickly select a scenario service and / or an atomic service according to personal needs, and through a user-selected device combination, a scenario service and / or an atomic service available under the device combination is provided in a targeted manner, and a user is listed with detailed and rich use scenarios, so as to guide the user to fully explore more application scenarios by using the functions of the super terminal, and an effect of recommending a scenario service according to a device combination is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0102] Figure 1 A schematic diagram of a communication system provided for an embodiment of the present application is shown in FIG. 1.

[0103] Figure 2 A hardware structure schematic diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 2.

[0104] Figure 3 A software architecture schematic diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 3.

[0105] Figure 4 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 4.

[0106] Figure 5 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 5.

[0107] Figure 6 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 6.

[0108] Figure 7 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 7.

[0109] Figure 8 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 8.

[0110] Figure 9 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 9.

[0111] Figure 10 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 10.

[0112] Figure 11 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 11.

[0113] Figure 12 A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 12.

[0114] Figure 13A A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 13.

[0115] Figure 13B A user interface schematic diagram provided for an embodiment of the present application is shown in FIG. 14.

[0116] Figure 13C A user interface schematic provided for an embodiment of the present application;

[0117] Figure 13D A user interface schematic provided for an embodiment of the present application;

[0118] Figure 14 A user interface schematic provided for an embodiment of the present application;

[0119] Figure 15 A user interface schematic provided for an embodiment of the present application;

[0120] Figure 16 A user interface schematic provided for an embodiment of the present application;

[0121] Figure 17 A user interface schematic provided for an embodiment of the present application;

[0122] Figure 18 A user interface schematic provided for an embodiment of the present application;

[0123] Figure 19 A user interface schematic provided for an embodiment of the present application;

[0124] Figure 20 A flowchart of a device interaction method provided for an embodiment of the present application;

[0125] Figure 21 A functional module schematic provided for an embodiment of the present application;

[0126] Figure 22A to Figure 22J Some user interface schematics provided for an embodiment of the present application;

[0127] Figure 23A to Figure 23D Some user interface schematics provided for an embodiment of the present application;

[0128] Figure 24A to Figure 24D Some user interface schematics provided for an embodiment of the present application;

[0129] Figure 25A to Figure 25H Some user interface schematics provided for an embodiment of the present application;

[0130] Figure 26A to Figure 26C Some user interface schematics provided for an embodiment of the present application;

[0131] Figure 27A to Figure 27E Some user interface schematics provided for an embodiment of the present application;

[0132] Figure 28A to Figure 28K Some user interface schematics provided for an embodiment of the present application;

[0133] Figure 29A and Figure 29B Some user interface schematic diagrams provided for the embodiments of the present application;

[0134] Figure 30A to Figure 30C Some user interface schematic diagrams provided for the embodiments of the present application;

[0135] Figure 31A to Figure 31C Some user interface schematic diagrams provided for the embodiments of the present application;

[0136] Figure 32A and Figure 32B Some user interface schematic diagrams provided for the embodiments of the present application;

[0137] Figure 33A to Figure 33C Some user interface schematic diagrams provided for the embodiments of the present application;

[0138] Figure 34A to Figure 34O Some user interface schematic diagrams provided for the embodiments of the present application. DETAILED DESCRIPTION

[0139] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone.

[0140] Hereinafter, the terms "first" and "second" are only used for description purposes, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features, and in addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0141] In the following embodiments of this application, the term "user interface (UI)" refers to the medium interface through which an application (APP) or operating system (OS) interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0142] With the upgrading of consumption, users own an increasing number of smart terminal devices. In some usage scenarios, users often need two or more terminal devices to work together or collaborate to provide certain services or achieve certain functions, which can also be called cross-device business. For example, mobile phones and smart screens can collaborate to cast videos; mobile phones can share files or pictures with other devices. In practical applications, it is found that when multiple devices work together, the user operation is cumbersome, the entry points are scattered, the human-computer interaction performance is poor, and the user experience is bad.

[0143] In some implementations, only a default service is provided when multiple devices work together. However, a single service is insufficient to meet the needs of diverse scenarios. For example, when a mobile phone and a smart screen collaborate, a multi-screen collaboration service can be provided. However, mobile phone and smart screen collaboration can also offer a wider range of services, such as services for home theater, motion-sensing games, and karaoke, but the mobile phone does not provide entry points for service options corresponding to these scenarios. If a user wants to switch to services for other application scenarios, they may need to exit the current interface, open a new application, and activate the corresponding service, which is cumbersome for the user.

[0144] This application provides a device interaction method that can be applied to a communication system composed of multiple devices. In some embodiments, this communication system may also be referred to as a super terminal or super terminal system. The purpose of this method is to explore novel human-computer interaction methods, provide more intuitive and convenient device interaction methods, meet users' needs for coordinating multiple devices, and provide users with entry points for service options corresponding to various scenarios.

[0145] In the following embodiments of the present application, a plurality of terminal devices can form a "super terminal" (or super terminal system). The "super terminal" refers to the integration of the capabilities of multiple terminals through distributed technology, stored in a virtual hardware resource pool, and the unified management, scheduling and integration of the capabilities of each terminal according to business needs to provide services to the outside, so that different terminals can achieve fast connection, capability assistance and resource sharing.

[0146] The super terminal can include a core device / central device, and a cooperative device / associated device. The core device is the control center of the super terminal system, and carries device management, service management, service scheduling and other functions. The core device is connected with one or more associated devices to form a super terminal. The associated device can be an electronic device located in the same communication network as the core device (such as a mobile phone), for example, an electronic device located in the same Wi-Fi network. The associated device can also be an electronic device whose login account is the same as the login account of the core device (such as a Huawei account), or an electronic device whose login account belongs to the same group (such as a same family account) as the login account of the core device. The associated device can also be an electronic device that establishes a trusted relationship (or pairing) with the core device through other means, such as a paired Bluetooth electronic device, an electronic device connected to a shared hotspot of the mobile phone, or an electronic device whose hotspot is connected by the mobile phone, or an electronic device that establishes a Wi-Fi P2P connection with the mobile phone, and the like. The embodiments of the present application do not limit the specific association method of the associated device and the core device.

[0147] Different associated devices and core devices are connected and work cooperatively to correspond to different functions. For example, the core device mobile phone and the associated device notebook computer form a super terminal, which can realize the screen projection function of projecting the mobile phone interface to the notebook computer; the core device mobile phone and the associated device smart speaker form a super terminal, which can realize the function of remotely controlling the smart speaker by the mobile phone, and the like.

[0148] It should be noted that in the following embodiments of the present application, "super terminal" is used to collectively refer to the communication system formed by the core device and one or more associated devices having an association relationship with the core device. That is, "super terminal" is used to express the set of electronic devices having the above association relationship. It can be understood that the "super terminal" is only an example of a descriptive word, and can be replaced by other words, such as "intelligent cooperation", "multi-device cooperation", etc., which does not constitute a specific limitation on the communication system in the embodiments of the present application. The present application does not limit the device types of the core device and the associated device.

[0149] The electronic device in the present application can be specifically a mobile phone, a tablet computer, a notebook computer, a personal computer (PC), a smart television (also referred to as a smart screen, a large screen, etc.), or a smart watch, a smart bracelet, and the like, and the present application does not make any limitation thereon. For example, taking a mobile phone as a core device / central device, the mobile phone can cooperate with one or more associated devices (such as a notebook computer, a PC, a tablet computer, a smart watch, and the like) to provide multi-functional, personalized, more interesting, and more convenient services for consumers.

[0150] The device interaction method provides a unified entrance for interaction between the core device and the plurality of associated devices, that is, a super terminal interface, and the user can trigger the super terminal composed of two or more electronic devices in the super terminal interface. Moreover, the super terminal system can support judging a use scenario according to the device type, device characteristics, product positioning, commonly used functions, device state, current running application, and current environment of each device when the plurality of devices initiate cooperation, and providing one or more scenario-based service options for the user based on the use scenario. Different scenario-based service options are entrances of corresponding services in different application scenarios, and the problem that the user cannot be provided with rich use scenarios in different device combinations is solved. In addition to supporting the electronic device to intelligently recommend scenario-based service options corresponding to a plurality of application scenarios for the user, the present application can also recommend different atomic service / function combinations or interaction function options according to different application scenarios or the type, characteristics, product positioning, and the like of different devices. In addition, the present application can also recommend matching devices for the user in order to expand more scenarios with better user experience.

[0151] Implementing the method provided by the present application can enhance the human-computer interaction performance, the user can quickly select scenario-based services and / or atomic services according to personal needs, without tedious and scattered settings, thereby the problem that the setting entrances of the interaction functions between electronic devices are scattered, the settings are tedious, and the user cannot be provided with rich use scenarios can be solved. By selecting the device combination of the user, the scenario-based services and / or atomic services available in the device combination are provided in a targeted manner, the user is listed with detailed and rich use scenarios, the user is guided to fully explore more application scenarios using the functions of the super terminal, and the effect of recommending scenario-based services according to device combinations is achieved. Moreover, a more intuitive and more concise user interface is presented to the user, the intuitiveness, readability, and operability of the user interface are improved, the user operation is simplified, the electronic device can quickly respond to open different application services, and the user experience is improved.

[0152] The communication system 10 provided by the embodiment of the present application is introduced below.

[0153] Figure 1 The communication system 10 provided by the embodiment of the present application is shown.

[0154] As Figure 1 shown, the communication system 10 includes a plurality of terminal devices, which can include the illustrated mobile phone 101, smart watch 102, notebook computer 103, tablet computer 104, smart bracelet 105, earphone 106, smart television (also known as smart screen, large screen, etc.) 107, smart speaker 108, and the like, in addition to Figure 1 In addition to the above devices shown, the communication system 10 can also include other types of electronic devices, such as personal computers (PCs), desktop computers, laptop computers, handheld computers, augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, car machines, game consoles, treadmills, cloud hosts / cloud servers, other smart wearable devices, and the like, and can also include internet of things (IOT) devices or smart home devices such as smart water heaters, smart lamps, smart air conditioners, smart body scales, and the like. The present application does not make any limitation on the type of device. In the present embodiment, the terminal device can also be referred to simply as a terminal, and the terminal device is generally a smart electronic device that can provide a user interface, can interact with a user, and can provide a service function for a user.

[0155] In the communication system 10, the mobile phone 101 as a central device / core device can establish a communication connection with other collaborative devices / associated devices to form a "super terminal". The "super terminal" refers to the integration of the capabilities of multiple terminals through distributed technology, storage in a virtual hardware resource pool, unified management, scheduling and integration of the capabilities of each terminal according to service requirements, and provision of services to the outside, so that fast connection, capability assistance and resource sharing are achieved between different terminals. For example, the mobile phone 101 and the notebook computer 103 form a super terminal, which can implement a screen projection function of projecting the interface of the mobile phone 101 onto the notebook computer 103; the mobile phone 101 and the smart speaker 108 form a super terminal, which can implement a function of remotely controlling the smart speaker 108 by the mobile phone 101, and the like.

[0156] It should be noted that in the following embodiments of the present application, "super terminal" is used to collectively refer to the mobile phone 101 as a core device and the associated devices associated with the mobile phone 101 to form a system. That is, "super terminal" is used to represent the collection of electronic devices having the above-mentioned associated relationship. It can be understood that the "super terminal" is only an example of a descriptive term, and can be replaced by other terms such as "intelligent collaboration", "multi-device collaboration", etc., which does not constitute a specific limitation on the communication system in the embodiments of the present application.

[0157] The communication connection between each terminal device in the communication system 10 can be a wired connection or a wireless connection, and the embodiments are not limited. Each terminal can transmit data or instructions through the established communication connection.

[0158] The communication connection can be a close-range communication connection. For example, a wired connection, such as a universal serial bus (USB) connection, a high-definition multimedia interface (HDMI) connection, a display port (DP) connection, and the like. Or a wireless connection, such as a Bluetooth (BT) connection, a wireless fidelity (Wi-Fi) connection, a hotspot connection, near field communication (NFC), ZigBee, and the like, to enable communication between each terminal without an account or with different accounts. Wireless connection has no connection constraints, and users have higher freedom of movement. The embodiments of the present application do not limit the type of communication connection. The terminal device can be configured with a Bluetooth (BT) module and / or a wireless local area network (WLAN) module. The Bluetooth module can provide one or more Bluetooth communication solutions including classic Bluetooth (Bluetooth 2.1) or Bluetooth low energy (BLE), and the WLAN module can provide one or more WLAN communication solutions including wireless fidelity peer-to-peer (Wi-Fi P2P), wireless fidelity local area networks (Wi-Fi LAN), or wireless fidelity software access point (Wi-Fi softAP). In some embodiments, Wi-Fi P2P refers to allowing devices in a wireless network to connect to each other in a point-to-point form without a wireless router, and in Wireless fidelity direct (Wi-Fi direct) in the system. Devices establishing a Wi-Fi P2P connection can exchange data directly through Wi-Fi (must be in the same frequency band) without connecting to a network or a hotspot, realizing point-to-point communication, such as transmitting files, pictures, videos, and the like. Compared with Bluetooth, Wi-Fi P2P has the advantages of faster search speed and transmission speed, longer transmission distance, and the like.

[0159] The communication connection can also be a long-distance communication connection. For example, each terminal logs in to the same account to realize connection and communication through the Internet. The plurality of electronic devices in the communication system 10 can also log in to different accounts, but are connected through binding. For example, the mobile phone 101 and the smart watch 102 can log in to different accounts, and the mobile phone 101 binds the smart watch 102 and itself in the device management application, and then connects through the device management application.

[0160] In addition, the plurality of terminal devices in the communication system 10 can also be connected and communicated in combination with any of the above manners, and the embodiments of the present application do not limit this. For example, the communication connection between the mobile phone 101 and the notebook computer 103 can be a combination of multiple connections, for example, the mobile phone 101 or the notebook computer 103 establishes a connection with the router through Wi-Fi to access the network, or establishes a connection with the base station through the cellular signal to access the network, and the mobile phone 101 and the notebook computer 103 can communicate through the network. For example, the mobile phone 101 sends information to the cloud server through the network, and the cloud server sends the information to the notebook computer 103 through the network.

[0161] In some embodiments, when two terminal devices are trusted devices for each other, for example, the two terminal devices have matched or connected before, when they are connected again, the two terminal devices will automatically establish a communication connection and then perform data interaction, without the need for the user to manually connect or match again, saving time and effort.

[0162] The terminal devices in the communication system 10 can be equipped with systems, systems, systems, HarmonyOS or other types of operating systems. The operating systems of the terminal devices in the communication system 10 can be the same or different, and the present application does not limit this. In some embodiments, each terminal connected in the communication system 10 is equipped with HarmonyOS, and the communication system 10 can be referred to as a HarmonyOS super virtual terminal (super virtual device) or a HarmonyOS super terminal.

[0163] In some embodiments, the mobile phone 101 or other terminal devices are installed with a super terminal application to facilitate the user to manage the terminal devices.

[0164] For example, the smart watch can detect the user's motion data, such as the number of steps when walking, the length of running, the length of swimming, and the like, and synchronize the user's motion data to the treadmill and / or the mobile phone. Similarly, the treadmill and / or the mobile phone can also synchronize the detected user data to the smart watch. The use of multiple terminals in cooperation can achieve more accurate detection of the user's motion data.

[0165] In the embodiment, the communication system 10 can analyze the use scenario of the current user based on the type and / or features of the currently connected devices, product positioning, commonly used functions, or according to the detected environment in which the user is currently located, or the running state of the device, and the like.

[0166] In the embodiment, the communication system 10 can comprehensively consider one or more scenario-based service options according to the type of the device in the system, the features of the device, the product positioning, the purpose of the device, the collaborative services that can be provided by the device, the environment in which the device is currently located, the running state of the device, the application that is recently used by the user, the currently running application, the orientation information of the user operation input, and the like, and display the scenario-based service options on the user interface to facilitate the user to quickly start the related services or functions.

[0167] In some embodiments, when listing the scenario-based service options, the scenario service providing module can calculate the probability of selection of each scenario-based service option according to the frequency of recent use of different collaborative services by the user, or the environment in which the current device is located, or the application that is recently run by the user, and display the scenario-based service options with a high probability of selection in the front in the scenario-based service option area.

[0168] In some embodiments, the communication system 10 can also analyze the current scenario based on the type of the device currently comprising the system, list other devices suitable for the current scenario to the user to help the user obtain a better scenario experience, or predict more expandable use scenarios, and then list other devices to the user. If the user adopts the devices recommended by the system, more and richer use scenarios can be further expanded.

[0169] In some embodiments, the communication system 10 can also automatically set the atomic service combination in the scenario based on the current device combination, the atomic services that can be provided by each device, the scenario-based service actually selected by the user, or based on the commonly used service combination of the user, or the default service combination set by the user, or the service combination most suitable for the scenario based on the life cognition, or the service combination with the most use frequency in the scenario obtained from the server, thereby saving the user from the tedious selection operation. The communication system 10 can also support the user to personally set the atomic service combination, and the user can manually modify the atomic service combination according to personal preferences or actual conditions, and adjust a certain atomic service to be provided by other devices.

[0170] The description of the specific embodiments can be understood with reference to the following examples.

[0171] It should be noted that, Figure 1 The communication system 10 shown is only used to assist in the description of the technical solutions provided by the embodiments of the present application, and does not constitute a limitation on other embodiments of the present application. In actual business scenarios, more or fewer terminal devices can be included in the communication system 10, and the present application does not make any limitation on the type, number, and connection mode of the terminal devices in the communication system 10.

[0172] The following describes an exemplary electronic device 100 provided by the embodiments of the present application.

[0173] Figure 2 The hardware structure schematic diagram of the electronic device 100 provided by the embodiments of the present application is shown. The exemplary electronic device 100 provided by the embodiments of the present application can be, but is not limited to, a mobile phone, a notebook computer, a tablet computer (portable android device, PAD), a smart bracelet, a smart watch, a headset, a personal computer (PC), a smart television, a smart speaker, etc. It can also be a desktop computer, a laptop computer, a handheld computer, an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a car machine, a game machine, a treadmill, a cloud host / server, other smart wearable devices, or an internet of things (IOT) device or a smart home device such as a smart water heater, a smart lamp, a smart air conditioner, a smart body scale, etc. Other types of electronic devices, and the present application does not make any limitation on the type of device.

[0174] It can be understood that the structure shown in the embodiment does not constitute a specific limitation on the first device. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, can increase or decrease part of the hardware structure, or combine certain components, or split certain components, or different component arrangement. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0175] Reference is made to Figure 2The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyro sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0176] The processor 110 is generally used to control the overall operation of the electronic device 100, and can include one or more processing units. For example, the processor 110 can include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a video processing unit (VPU), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated into one or more processors. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0177] The digital signal processor is used to process digital signals, and in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 is in frequency point selection, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0178] Video codecs are used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record video in a variety of encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, and so on.

[0179] An NPU is a neural-network (NN) computing processor that rapidly processes input information by drawing on the structure of a biological neural network, such as the transmission mode between human brain neurons, and can also constantly self-learn. Through an NPU, intelligent cognitive applications of an electronic device can be implemented, such as: image recognition, facial recognition, voice recognition, text understanding, and so on.

[0180] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or is repeatedly using. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0181] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, a serial peripheral interface (SPI) interface, and so on.

[0182] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can contain multiple sets of I2C bus. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces respectively. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface, and realize the touch function of the electronic device.

[0183] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can contain multiple sets of I2S bus. The processor 110 can be coupled to the audio module 170 through the I2S bus, and realize the communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface.

[0184] The PCM interface can also be used for audio communication, which samples, quantizes and encodes analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through the PCM bus interface. In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface. Both the I2S interface and the PCM interface can be used for audio communication.

[0185] The UART interface is a universal serial data bus, which is used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.

[0186] In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface, and realizes the Bluetooth function. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the UART interface, and realize the function of playing audio.

[0187] The MIPI interface can be used to connect the processor 110 and the display screen 194, the camera 193, and other peripheral devices. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the photographing function of the electronic device. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device.

[0188] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0189] The USB interface 130 is an interface that conforms to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device, or to transmit data between the electronic device and a peripheral device. The interface can also be used to connect other electronic devices, such as a mobile phone, a PC, a smart TV, and the like. The USB interface can be USB 3.0, which is used to transmit display port (DP) signals and can transmit high-speed audio and video data.

[0190] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods.

[0191] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of the electronic device. The charging management module 140 can charge the battery 142 while also providing power to the electronic device through the power management module 141.

[0192] The power management module 141 is configured to connect the battery 142 and the charging management module 140 to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), and the like. In some embodiments, the power management module 141 can also be disposed in the processor 110. In some embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0193] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, and the like.

[0194] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be configured to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some embodiments, the antennas can be used in combination with a tuning switch.

[0195] The mobile communication module 150 can provide a solution for wireless communication including a second generation (2th generation, 2G) network / third generation (3th generation, 3G) network / fourth generation (4th generation, 4G) network / fifth generation (5th generation, 5G) network, and the like, applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, and the like on the received electromagnetic waves, and transfer the processed electromagnetic waves to the modem processor to be demodulated. The mobile communication module 150 can also amplify signals modulated by the modem processor, and radiate the amplified signals as electromagnetic waves through the antenna 1. In some embodiments, at least part of the function modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the function modules of the mobile communication module 150 and at least part of the modules of the processor 110 can be disposed in the same device.

[0196] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a microphone 170B, etc.), or displays an image or a video through the display 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110 and disposed in the same device as the mobile communication module 150 or other functional modules.

[0197] The wireless communication module 160 can provide a wireless communication solution including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification, and convert the signal into electromagnetic wave radiation via the antenna 2.

[0198] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0199] The electronic device 100 can implement a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0200] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0201] The electronic device 100 can implement a photographing function through an ISP, the camera 193, a video codec, a GPU, the display screen 194, and an application processor, and the like.

[0202] The ISP is configured to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize algorithms for noise, brightness, and skin color of the image. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0203] The camera 193 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard image signal in a format such as RGB, YUV, etc. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1. The camera 193 can include, but is not limited to, a conventional color camera (RGB camera), a depth camera (RGB depth camera), a dynamic vision sensor (DVS) camera, etc. In some embodiments, the camera 193 can be a depth camera. The depth camera can capture spatial information of a real environment.

[0204] In some embodiments, the electronic device 100 can capture images of different users through the camera 193, recognize the different users by facial recognition, and enable different user accounts corresponding to the different users. The accounts of the different users can store information of the different users, so that the accounts of the different users can be prevented from being confused, and the data privacy of the users can be further protected.

[0205] In some embodiments, the camera 193 can capture images of hands or bodies of the user, and the processor 110 can be configured to analyze the images captured by the camera 193, so as to recognize hand actions or body actions input by the user. For example, the camera 193 can be configured to recognize hand actions of the user, so as to realize gesture control of the user.

[0206] The internal memory 121 can be configured to store computer executable program codes including instructions. The processor 110 can execute various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data (such as audio data, a phone book, etc.) created during use of the electronic device, etc.

[0207] In some embodiments of the present application, the internal memory 121 can be used to store an application program of one or more applications, which includes instructions. When the application program is executed by the processor 110, the electronic device 100 is caused to generate content for presentation to a user. Exemplarily, the application can include an application for managing the electronic device 100, such as a game application, a meeting application, a video application, a desktop application, or other applications, and the like.

[0208] The internal memory 121 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs).

[0209] The random access memory has the characteristics of fast read / write speed and volatility. Volatility refers to the fact that the data stored in the RAM will disappear once the power is off. Generally, the static power consumption of the random access memory is extremely low, and the running power consumption is relatively large. The data in the RAM is memory data, which can be read at any time and disappears when the power is off.

[0210] The non-volatile memory has the characteristics of non-volatility and stable storage of data. Non-volatility refers to the fact that the data stored therein will not disappear after power off, and can be saved for a long time. The data in the NVM includes application data, which can be stably stored in the NVM for a long time. Application data refers to the content written during the running of an application program or a service process, such as photos or videos obtained by a photo application, text edited by a user in a document application, and the like.

[0211] The random access memory can include a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), and the like.

[0212] The non-volatile memory can include a magnetic disk storage, a flash memory, and the like.

[0213] Disk storage devices are storage devices that use disks as storage media. They are characterized by large storage capacity, high data transfer rate, and long-term data preservation.

[0214] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0215] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0216] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0217] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0218] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0219] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0220] The speaker 170A, also called a "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device can listen to music or listen to a hands-free call through the speaker 170A.

[0221] The receiver 170B, also called a "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device answers a call or a voice message, the user can listen to the voice by holding the receiver 170B close to the ear.

[0222] The microphone 170C, also called a "microphone", "sounder", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can make a sound by holding the microphone 170C close to the mouth, and input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, in addition to collecting sound signals, it can also realize the function of noise reduction. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C, to realize the collection of sound signals, noise reduction, and also to identify the source of the sound, to realize the function of directional recording, etc.

[0223] The earphone interface 170D is used to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0224] The electronic device 100 can include one or more keys 190, which can control the electronic device 100 and provide the user with access to the functions on the electronic device 100. The form of the key 190 can be a button, a switch, a dial, etc. mechanical case, but also can be a touch or near touch type sensing device (such as a touch sensor). The electronic device 100 can receive a key input, generate a key signal input related to the user settings and function control of the electronic device 100. The key 190 can include a power-on key, a volume key, etc.

[0225] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompt, and also can be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations acting on different regions of the electronic device 100 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, receiving messages, alarms, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0226] The indicator 192 can be an indicator light, which can be used to indicate the charging state, the power change, and also can be used to indicate messages, notifications, etc.

[0227] The electronic device 100 can also include other input / output interfaces, and other devices can be connected to the electronic device 100 through appropriate input / output interfaces. Components can include, for example, audio / video jacks, data connectors, etc.

[0228] The electronic device 100 is equipped with one or more sensors, including but not limited to a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0229] The pressure sensor 180A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display 194. The pressure sensor 180A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the force based on the change in capacitance. When a touch operation is applied to the display 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed. In some embodiments, touch operations applied to the same touch position but with different touch operation time lengths can correspond to different operation instructions. For example, when a touch operation with a time length less than a first time threshold is applied to the pressure sensor 180A, an instruction to confirm is executed. When a touch operation with a time length greater than or equal to the first time threshold is applied to the pressure sensor 180A, an instruction to turn on / off the electronic device 100 is executed.

[0230] The gyroscope sensor 180B can be configured to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake photography. Illustratively, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device shaking, calculates the distance that the lens module needs to compensate based on the angle, and lets the lens offset the shaking of the electronic device by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0231] The barometric pressure sensor 180C is configured to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the barometric pressure sensor 180C, and assists in positioning and navigation.

[0232] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the opening and closing of the flip cover can be detected by the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0233] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0234] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0235] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object nearby. The electronic device 100 may use the proximity sensor 180G to detect when a user holds the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0236] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0237] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0238] The temperature sensor 180J is configured to detect temperature. In some embodiments, the electronic device 100 performs a temperature handling strategy based on the temperature detected by the temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In another example, when the temperature is below another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by low temperature. In yet another example, when the temperature is below yet another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0239] The touch sensor 180K is also referred to as a "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also referred to as a "touch panel". The touch sensor 180K is configured to detect a touch operation acting on or near the touch sensor 180K. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In another example, the touch sensor 180K can also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0240] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a human body sound vibration bone block. The bone conduction sensor 180M can also contact the human body pulse to receive a blood pressure pulsation signal. In some embodiments, the bone conduction sensor 180M can also be disposed in a headset to form a bone conduction headset. The audio module 170 can analyze a voice signal based on the vibration signal of the sound vibration bone block obtained by the bone conduction sensor 180M to implement a voice function. The application processor can analyze heart rate information based on the blood pressure pulsation signal obtained by the bone conduction sensor 180M to implement a heart rate detection function.

[0241] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0242] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture. Taking the system as an example, the software structure of electronic device 100 is illustrated.

[0243] Figure 3 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0244] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, this can be... The system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime layer, and the application application layer. runtime and system libraries, as well as the kernel layer.

[0245] The application layer can include a series of application packages.

[0246] like Figure 3 As shown, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and settings. The settings application allows users to adjust font size, weight, and other settings.

[0247] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0248] like Figure 3As shown, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0249] The window manager is used to manage windows. The window manager can acquire the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen, etc.

[0250] The content provider is used to store and acquire data, and make the data accessible to the application. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phonebook, etc.

[0251] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0252] The phone manager is used to provide communication functions. For example, management of call status (including connection, hang-up, etc.).

[0253] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.

[0254] The notification manager enables the application to display notification information in the status bar, which can be used to convey messages of the notification type, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloading, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, or a notification in the form of a dialog window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, vibrating the electronic device, flashing the indicator light, etc.

[0255] The runtime includes the core library and the virtual machine. The runtime is responsible for the scheduling and management of the Android system.

[0256] The core library contains two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0257] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection, etc.

[0258] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), etc.

[0259] The surface manager is used to manage a display subsystem and provides fusion of 2D and 3D layers for a plurality of applications.

[0260] The media libraries support a plurality of commonly used audio, video format playback and recording, and static image files, etc. The media libraries can support a plurality of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0261] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0262] The 2D graphics engine is a drawing engine for 2D drawing.

[0263] The kernel layer is a layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, and sensor drivers.

[0264] The above description of the software architecture of the electronic device 100 is merely an example. It can be understood that the software architecture illustrated by the embodiments of the present application does not constitute a specific limitation on the present application. In other embodiments of the present application, the software architecture of the electronic device 100 can include more or fewer modules than the illustration, or combine certain modules, or split certain modules, or different architectural arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0265] The technical solutions of the present application are described below in conjunction with some embodiments providing related illustrative user interfaces.

[0266] After the core device (such as a mobile phone) and the associated device (such as a smart watch) establish a communication connection, the core device and the associated device can transmit instructions, data, etc. to each other through the communication connection. The communication connection can be a USB connection, a Bluetooth connection, a Wi-Fi P2P connection, etc., and the embodiments are not limited.

[0267] In the following embodiments of the present application, a mobile phone is taken as a core device, and other devices such as a smart watch, a smart television, etc. are taken as associated devices connected with the mobile phone to form a super terminal as an example for description, which does not constitute any limitation on the device types of the core device and the associated device in actual applications.

[0268] The same account can be logged in on the core device and the associated device, or different accounts can be logged in, and the embodiments are not limited.

[0269] The core device is a control center of the super terminal system, and carries functions of device management, service management, service scheduling, etc.

[0270] The associated device can be an electronic device located in the same communication network as the core device (such as a mobile phone), for example, an electronic device located in the same Wi-Fi network. The associated device can also be an electronic device whose login account is the same account as the login account of the core device (such as a Huawei account), or an electronic device whose login account belongs to the same group (such as a same family account) as the login account of the core device. The associated device can also be an electronic device that establishes a trusted relationship (or pairing) with the core device through other ways, such as a paired Bluetooth electronic device, an electronic device that has connected to a hotspot shared by the mobile phone, or an electronic device whose hotspot has been connected by the mobile phone, or a mobile phone that has established a Wi-Fi P2P connection with the electronic device. The embodiments of the present application do not limit the specific association method of the associated device and the core device.

[0271] It can be understood that in the following example interface provided by the embodiments of the present application, the core device is mainly a mobile phone to display the related user interface, but this does not limit the present application. The following example interface can also be migrated to other types of devices, as long as the same invention idea provided by the present application is based on, it is within the protection scope of the present application.

[0272] It can be understood that the various user interfaces described in the following embodiments of the present application are only example interfaces, and do not limit the present application scheme. In other embodiments, different interface layouts can be used in the user interface, more or fewer controls can be included, and other function options can be added or reduced, as long as the same invention idea provided by the present application is based on, it is within the protection scope of the present application.

[0273] Reference Figure 4 The embodiments shown, Figure 4 It is shown that the user can enter the super terminal interface through the pull-down control center interface.

[0274] As Figure 4 shown, the user interface 400 is a schematic diagram of a main interface (or a main screen interface, etc.) displayed by the mobile phone. The user interface 400 can include a top status bar 401, a desktop 402, a bottom tray bar 403, etc. Among them, the top status bar 401 includes a mobile signal indicator, a wireless network signal indicator, a power indicator, a time indicator, etc. The desktop 402 can display one or more application icons, application function components, etc. The bottom tray bar 403 can display icons of commonly used application programs to facilitate the user to quickly open these applications. In the example of the user interface 400, the bottom tray bar 403 displays icons of the phone, the message, the address book, and the camera application programs.

[0275] As shown in FIG. 4A, a user can invoke the control center interface through a user operation 404. Figure 4 The user operation 404 shown in FIG. 4A can be a touch operation in which the user performs a downward swipe from the top of the display interface. In other embodiments, the user operation 404 can also be a touch operation in which the user performs an upward swipe from the bottom of the display interface. Figure 4

[0276] After detecting the user operation 404, the phone displays the control center interface, as shown in user interface 410. The user interface 410 can display a plurality of function controls, which can also be referred to as cards or modules, etc. For example, the user interface 410 shown in FIG. 4A includes a date and time display control 411, a settings control 412, a quick settings card 413, a super terminal function card 414, a music card 415, a desk lamp quick control card 416, and a smart screen quick control card 417.

[0277] The date and time display control 411 displays the current time and date. The settings control 412 can be used by the user to quickly enter the settings interface. The quick settings card 413 includes a plurality of controls, which can be used by the user to quickly turn on / off or adjust corresponding functions, such as quickly turning on / off a Wi-Fi network, Bluetooth, a mobile network, sound, an alarm, and quickly adjusting the screen display brightness, etc. The super terminal function card 414 displays one or more device icons, which can include devices that have been connected to the phone to form a super terminal, or devices that have not been connected to the phone, which can be devices that have been previously paired or devices that have not been paired, and the present embodiment does not limit this. When the user clicks the super terminal function card 414, the phone can display the user interface 500 shown in FIG. 4B. The music card 415 can display information such as the currently playing music and the singer, and also includes controls such as the previous song, play / pause, next song, and song list. The desk lamp quick control card 416 can be used by the user to quickly turn on / off a desk lamp. The smart screen quick control card 417 can be used by the user to quickly turn on / off a smart screen. Figure 5

[0278] In other embodiments, the user can also enter the super terminal interface by selecting a super terminal option in the settings page, that is, the user operation 404 can include a series of operations, such as an operation in which the user clicks an application icon of the “settings” application in the main interface, and an operation in which the user selects a corresponding function option in the “settings” application interface, etc. Alternatively, the user can enter the super terminal interface from an entry such as the minus one screen of the phone, the upper pull-down toolbar, the lower pull-down notification bar, the left / right pull quick icon bar, the desktop card, or an application such as a smart home application, etc., and the present embodiment does not limit the way in which the user enters the super terminal interface. ​​

[0279] When the user clicks on the HyperTerminal function card 414, the phone can display... Figure 5 The user interface shown is 500. Figure 5 It is the device interaction interface related to the SuperTerminal function.

[0280] In some embodiments, the mobile phone can activate its wireless communication function, search for nearby electronic devices, and display the names of the found electronic devices in a device list. The user then selects the electronic device to interact with from the device list by name and initiates a connection. In some embodiments, unless otherwise specified by the user, the names of the electronic devices displayed in the phone's device list are typically the type and / or model of the electronic device, such as HUAWEI Vision or HUAWEI MateBook. Sometimes, two or more electronic devices with the same name may appear, making it difficult for the user to determine which device corresponds to which device.

[0281] This application provides a novel interactive interface that offers users a more intuitive interface and a more user-friendly and intuitive interactive experience, such as... Figure 5 The HyperTerminal User Interface 500 shown is shown.

[0282] The user interface 500 may display a HyperTerminal title bar 501, a return control 502 for returning to the previous page, prompts 503, a device connection status diagram 504, etc.

[0283] In the 503 error message, the text "Drag and drop the device to this machine to establish collaboration" is displayed.

[0284] The device connection status icon 504 can be represented as a pie chart. A mobile phone icon 505 is displayed at the center of the icon 504, indicating that the core device is a mobile phone. The device icons and names surrounding the mobile phone represent the nearby devices that the mobile phone has found, such as... Figure 5 The icons shown include smartwatch icon 506, Sound X icon, Vision icon, FreeBuds icon, smart bracelet icon, MatePad icon, MateBook icon, etc.

[0285] It can be noted that in the device connection state diagram 504, the layout relationship between the core device and the associated device can be indicated visually. As the relative positions of the associated device icon and the core device icon shown in the device connection state diagram 504, the position relationship of the associated device and the core device in the real space, such as relative orientation information, relative distance information, etc., can help the user quickly find and distinguish each electronic device. For example, the mobile phone searches for two devices named smart watch, and the user can distinguish the device to be selected in the super terminal interface of the mobile phone according to the relative position (relative orientation or distance) of the identified device to the mobile phone. For example, in the real space, smart watch A is located on the left side of the mobile phone, and smart watch B is located on the right side of the mobile phone. In the device connection state diagram, the user can distinguish that the smart watch icon on the left side of the mobile phone icon represents smart watch A, and the smart watch icon on the right side of the mobile phone icon represents smart watch B.

[0286] In some embodiments, in the device connection state diagram 504, the distances of different associated devices to the core device mobile phone can also be different, and the distances between the associated device icons and the core device icons in the interface can indicate the distances in the real space, such as the farther the distance between the associated device and the core device in the real space, the farther the distance between the associated device icon and the core device icon displayed in the interface. The specific technology for implementing this function is not limited in this embodiment, such as the mobile phone can obtain the orientation of the directional information carried by the associated device based on the ultrawideband (ultrawideband, UWB) technology and the measured distance, and obtain the spatial mapping relationship between the associated device and the mobile phone in the real space.

[0287] In other embodiments, the distance between the associated device icon and the core device icon in the interface can indicate the signal strength of the associated device detected by the core device, such as the Bluetooth signal strength or the WiFi signal strength, for example, the stronger the signal strength of the associated device detected by the core device, the closer the distance between the associated device icon and the core device icon displayed in the interface.

[0288] The layout, form of presentation and indication meaning of the device connection state diagram 504 are not limited in this application. In other embodiments, other layouts or forms of presentation can also be used to indicate the detected associated devices.

[0289] In some embodiments, the mobile phone can periodically and automatically detect nearby associated devices. For example, the phone can display a radar scan-like interface to indicate that the phone is currently performing a periodic detection task. Alternatively, the phone can respond to user actions on a refresh control and then detect nearby associated devices. For instance, the phone can use its wireless communication capabilities to broadcast a query message to search for associated devices in the vicinity. When a nearby electronic device receives the query message from the phone, it can return a response message to the phone, which may carry information such as the electronic device's media access control address (MAC address), device type, and name.

[0290] A mobile phone can determine whether an electronic device is associated with it, i.e., whether the electronic device is an associated device of the mobile phone, based on response messages from other electronic devices. For example, the mobile phone can use the MAC address of the electronic device carried in the response message to query the account management server to determine whether the account logged in by the electronic device is the same as the account logged in by the mobile phone, or whether the accounts belong to the same group. Alternatively, the mobile phone can use the MAC address of the electronic device carried in the response message to query the router of the Wi-Fi network where the electronic device is located to determine whether the electronic device is on the same Wi-Fi network as the mobile phone. Or, the mobile phone can use its stored historical records to determine whether the electronic device has established a trusted relationship with the mobile phone. This application does not limit the specific method for determining the associated devices of a mobile phone.

[0291] The aforementioned wireless communication functions include, but are not limited to, wireless communication functions implemented by technologies such as WLAN, radio frequency identification (REID), Wi-Fi P2P, hotspot, infrared, ultrasound, Bluetooth, ZigBee, and UWB. This embodiment does not limit these functions.

[0292] Understandably, many wireless communication technologies support positioning functions. Examples include Wi-Fi-based positioning, Bluetooth iBeacon-based positioning, Bluetooth 5.1-based positioning using the angle of arrival (AOA), and UWB-based positioning, etc., which are not limited in this embodiment. Among these, Wi-Fi-based and Bluetooth iBeacon-based positioning functions can measure the distance between the mobile phone and other electronic devices. Bluetooth 5.1-based AOA positioning and UWB-based positioning functions can measure the distance between the mobile phone and other electronic devices, as well as the orientation of other electronic devices relative to the mobile phone.

[0293] After the mobile phone searches for nearby connected devices, it can further utilize the positioning capabilities of its wireless communication function to measure the relative direction and / or relative distance between the connected devices and the mobile phone in real space, and then display the result as shown below. Figure 5 The device connection status diagram 504 is shown. The layout of the icons (and names) representing each electronic device in the device connection status diagram 504 can be consistent with the actual spatial layout of each electronic device. For example, in the embodiment provided in this application, Figure 5 The diagram illustrates the positional relationships (distance relationships not shown) between multiple associated devices (such as smartwatch 102, laptop 103, tablet 104, smart bracelet 105, earphones 106, smart TV 107, and smart speaker 108) and mobile phone 101. Figure 1 The layout of each associated device icon in the device connection status diagram 504 corresponds one-to-one with the phone icon. For example, Figure 5 As shown, the smartwatch 102 is located directly in front of the phone 101, and correspondingly, Figure 1 The device connection status diagram 504 shows the smartwatch icon 506 directly above the phone icon 505.

[0294] Figure 5 The icons for associated devices are positioned outside the core device, not attached to, overlapping with, or aligned with it, indicating that these associated devices have not yet established a connection with the core device. In other words, while the core device is detecting which associated devices are nearby, both the phone and each associated device need to have wireless communication enabled, but a pre-existing wireless connection is not required.

[0295] If a user wants a connected device to form a super terminal with the core device (mobile phone) to achieve certain collaborative functions, in one implementation method, the user can... Figure 5 On the HyperTerminal interface shown, select the associated device icon and drag the corresponding icon closer to the core device icon, for reference. Figure 5 The user interface shown is 600.

[0296] Assuming that before the devices are synchronized, the distance between the associated device icon and the core device icon is a first distance, when the associated device icon is dragged close to the core device icon, and the distance between them is less than or equal to a second distance, the user releases their finger, and the associated device icon will snap to the vicinity of the core device icon. (See reference...) Figure 6 The user interface shown is 700.

[0297] like Figure 7As shown, if a user wants the smartwatch and mobile phone to form a super terminal and work together, the user can press and hold the smartwatch icon 506 and drag the smartwatch icon 506 closer to the mobile phone icon 505. The user interface 600 can also display a prompt 602 "Smartwatch selected," indicating the current user selection status.

[0298] When the distance between the smartwatch icon 506 and the phone icon 505 is less than or equal to the second distance, or when the smartwatch icon 506 enters a circular area with the center of the phone icon 505 as the center and a radius of the first radius distance R, the user can release their finger to display... Figure 6 In the device connection status diagram 701 shown, the smartwatch icon 702 is attached to the edge of the phone icon 505. The user interface 700 can also display a prompt 703 "Smartwatch is working together," indicating the current connection status of the HyperTerminal.

[0299] Of course, if the smartwatch has already established a connection with the phone before opening the HyperTerminal interface, then when the phone detects the established connection, it will directly display the interface where the smartwatch icon 702 is attached to the edge of the phone icon 505, indicating that the smartwatch has established a connection with the phone. Figure 7 The user interface shown is 700.

[0300] Understandably, the ability to establish a communication connection with a mobile phone is a necessary but not sufficient condition for a connected device. A device that can establish a communication connection with a mobile phone is not necessarily a connected device capable of forming a super terminal; however, a connected device must be a device capable of establishing a communication connection with a mobile phone.

[0301] exist Figure 7 In the diagram, the smartwatch icon 702, which has been connected to the phone to form a super terminal, can display a distinctive identifier to easily distinguish it from the icons of associated devices that are not connected. For example, the smartwatch icon 702 is displayed with a dark fill, indicating that the smartwatch is connected to the phone. Other features include the smartwatch icon 702 being smaller than the smartwatch icon 506 when not connected, the circular icon of the smartwatch icon 702 having a diameter of R2, and being snapped onto the edge of the phone icon 505, with the edges of the smartwatch icon 702 and the phone icon 505 being tangent.

[0302] In other embodiments, "two device icons snapping together" can also mean that the two device icons are wholly or partially overlapping. This embodiment does not limit the form of snapping.

[0303] Correspondingly, if the user wants to let a certain associated device leave the super terminal system, the user can select the associated device icon and drag the associated device icon away from the core device icon when the associated device icon and the core device icon are in the state of being attracted. When the mobile phone detects that the user drags the associated device icon away from the core device icon by more than a third distance, the associated device icon returns to the position before being connected when the user releases the finger, the state of being attracted is released, the associated device is also disconnected from the core device, leaves the super terminal system, and the services provided by the device in the super terminal are also terminated. In some examples, the core device icon remains stationary, the user selects the associated icon and drags the associated icon away from the core device icon by more than a certain distance, and the two devices are disconnected. In other examples, the user can select the associated icon and the core device icon at the same time, hold the two icons respectively, and drag the two icons in opposite directions by more than a certain distance, and the two devices are disconnected.

[0304] The embodiment does not limit the conditions or user operations for triggering attraction and releasing attraction.

[0305] In other embodiments, when the user drags the associated device icon close to the core device icon by less than the second distance, the user does not release the associated device icon, but the associated device icon can be attracted to the core device icon after staying at the position for more than a first preset time length. Correspondingly, the user can also drag the associated device icon away from the core device icon by more than the third distance, the user does not release the associated device icon, but the associated device icon returns to the position before being connected, the state of being attracted is released, the associated device is also disconnected from the core device, leaves the super terminal system, and the services provided by the device in the super terminal are also terminated after staying at the position for more than a second preset time length.

[0306] Alternatively, in other embodiments, when the user clicks on the icon of a certain associated device in the unconnected state, the connection between the associated device and the core device can be triggered, and the associated device icon and the core device icon are attracted together on the interface. Correspondingly, when the user clicks on a certain associated device in the connected state, the disconnection between the associated device and the core device can be triggered, and the associated device icon and the core device icon are separated, and the associated device icon returns to the position in the unconnected state.

[0307] In some embodiments, the selected device icon can change its position along with the trajectory of the user's finger dragging on the screen. The present embodiments do not limit the trajectory of the user's finger dragging the device icon, and the user's dragging trajectory can be any curve. For example, after the user selects a certain associated device icon, the user drags the associated device icon on the screen for a distance or stops at a certain position on the screen for a second preset time length, the phone can detect whether the distance between the associated device icon and the phone icon is less than or equal to a certain distance, such as the second distance, and if so, the associated device icon and the phone icon are attracted together. When the user's finger moves, the associated device icon moves along with the trajectory of the user's finger.

[0308] In some other embodiments, the selected device icon can also not change its position along with the trajectory of the user's finger dragging on the screen. For example, after the user selects a certain associated device icon, the user drags the associated device icon on the screen for a distance or stops at a certain position on the screen for a second preset time length, the phone can determine whether the distance between the current position of the user's finger and the phone icon is less than or equal to a certain distance, such as the second distance, and if so, the associated device icon and the phone icon are attracted together. When the user's finger moves, the associated device icon does not move.

[0309] In some other embodiments, the user can also select the core device icon close to the associated device icon, which also triggers the connection of the two devices. For example, the user holds the phone icon and drags the phone icon close to the smart watch icon, and when the phone icon and the smart watch icon are close or have an overlapping area, the user releases the finger, and the smart watch icon can also be attracted to the phone icon, indicating that the phone and the smart watch are connected and can work together.

[0310] In some embodiments, a one-key disconnecting control can also be provided on the super terminal interface, and clicking the one-key disconnecting control can disconnect all associated devices from the core device, restore the core device to a state of being independent and having no associated devices connected, and the services provided by the associated devices are also terminated.

[0311] It can be understood that the connection between the associated device and the core device to form the super terminal is not limited, and in some other embodiments, the user can also connect a certain associated device with another associated device in the super terminal interface to form a super terminal. That is, the user can connect any two or more devices, and the devices having a connection relationship are not limited to being associated devices or core devices, such as the associated device smart speaker and the smart TV, which can be directly connected to form a super terminal, and various functions, schematic interfaces or user operations thereof can refer to the descriptions in the foregoing or subsequent embodiments, which will not be described in detail here.

[0312] In the embodiments provided in the present application, when multiple devices are connected to form a super terminal, the super terminal system can analyze the use scenario of the current user based on the types and / or characteristics of the devices that have been connected, or according to the environment in which the user is currently located, or the running state of the devices, and the like. Further, the super terminal system can list the scenario-based services or functions that it can provide, and display them on the user interface, so as to facilitate the user to quickly start the related services or functions.

[0313] Specifically, different device types have different device characteristics. For example, a smart TV has a large screen, and the user's visual viewing experience is good, and it is more suitable for watching videos; a smart speaker has strong audio output capability, and the sound effect is good, and it is more suitable for playing audio; an earphone is small in size, easy to carry, does not need to be played externally, does not disturb people around, and can also have a noise reduction effect, shield external environmental noise, and is suitable for listening to audio in public places, such as making a call or listening to music in public places; a smart watch or a smart bracelet can more accurately measure the motion data or heart rate and other health information of the user than a mobile phone; a notebook computer has strong processing capability, and is more suitable for office work. Here, the device types and device characteristics will not be described again, and the above examples do not limit other embodiments.

[0314] Alternatively, the device can also detect the environment in which the user is currently located, and analyze the use scenario of the current user. The present embodiment does not limit the way of detecting the environment in which the user is located. For example, the current positioning information of the device can be used to determine whether it is indoors or outdoors; or a camera can be used to collect a photo of the current environment, or ultrasonic waves, infrared rays, and the like can be used to collect the spatial depth or size of the current environment, so as to determine whether the current user is indoors or outdoors.

[0315] In some embodiments, the super terminal system can divide multiple consumer use scenarios. The super terminal system can match the related use scenarios based on the device types, device characteristics, product positioning, commonly used functions, and the like of the devices in the current system.

[0316] In one example, the consumer use scenarios can include the following five categories: mobile office, smart home, audio-visual entertainment, sports health, and smart travel. Of course, the classification of the consumer use scenarios is not fixed, and the super terminal system can continuously update, expand or refine more use scenarios, and the present embodiment is not limited, and can be changed according to actual conditions. For example, if a new associated device or function is added that has not appeared before, it can lead to the appearance of a new use scenario. For example, a smart car is added as an associated device that can form a super terminal with a mobile phone, and then a new use scenario of the mobile phone and the smart car is added, which is a smart travel scenario.

[0317] After the super terminal system matches the device with the use scene, the device combination can provide a scene service, an atomic service, etc. based on the use scene.

[0318] The scene service refers to a functional service based on a use scene, and is a collection of multiple functions or services provided for a special scene. The atomic service refers to a minimum capability unit that can be independently run, and is a concept of abstract encapsulation of a single function / capability, which can be a hardware service or a software service. Generally, a scene service can be supported by multiple atomic services, and an atomic service combination includes atomic services with atomic capabilities of multiple devices. The atomic capabilities can include one or more of the following: audio output capability, audio input capability, display capability, camera capability, touch input capability, and keyboard / mouse input capability. For example, the audio output capability can include support for single-channel or multi-channel, supported sound effects, supportable frequency response range, noise reduction capability, and audio resolution capability when the device plays audio; the audio input capability can include sound collection range and noise reduction capability; the display capability can include screen size, display resolution parameters, refresh rate, and color performance of the device; the camera capability can include camera type, shooting pixels, night scene shooting capability, and image adjustment capability of the device; and the touch input capability and the keyboard / mouse input capability refer to whether the device can support touch input or keyboard / mouse input.

[0319] For example, the multi-screen collaboration scene service between a mobile phone and a large screen is supported by the following atomic services: display service, audio input service, audio output service, touch input service, keyboard / mouse input service, and camera service. The display service can be provided by the large screen, the audio input service can be provided by the mobile phone, the audio output service can be provided by the large screen, the touch input service can be provided by the mobile phone, and the keyboard / mouse input service can be provided by the large screen.

[0320] It can be understood that the scene service and the atomic service are only words used in the embodiment, and the meanings represented by the words have been described in the embodiment, and the names do not constitute any limitation on the embodiment. For example, in other embodiments, the scene service can also be referred to as a scene function, a scene service, a scene business, a business function, or other terms; and the atomic service can also be referred to as a meta capability, an atomic capability (Ability), an atomic service, a functional component, or other terms. In the embodiment of the application, the scene service and the atomic service are mainly described.

[0321] Taking a super terminal composed of a mobile phone and a smart watch as an example, a use scenario that can be matched is a sports health scenario. The super terminal system can query the scenario-based services and atomized services that can be provided by the combination of the mobile phone and the smart watch. For example, the combination of the mobile phone and the smart watch can provide services including supporting sports services, supporting health services, counting exercise, measuring heart rate, measuring blood oxygen saturation, and the like. In combination with different scenarios, the scenario-based services recommended for the combination of the mobile phone and the smart watch include indoor running, outdoor running, heart health, blood oxygen health, and the like. The atomized services supported by the smart watch include GPS positioning, sound output, vibration reminder, and the like.

[0322] In one example, referring to Table 1, Table 1 lists the core device as a mobile phone, some different associated devices that form a super terminal with the mobile phone, the corresponding classified use scenario categories, and the scenario-based services recommended after querying the device capabilities. It should be noted that Table 1 only shows the case where the associated device is a single device, but is not limited thereto. In other embodiments, there are also cases where multiple associated devices form a super terminal with the mobile phone, for example, the associated devices are a smart television and a smart speaker, and the corresponding execution operations and function descriptions can refer to the foregoing or the following embodiments, which will not be described here.

[0323] Table 1

[0324]

[0325] The scenario-based services can be scenario-based service options obtained by the super terminal system through comprehensive statistics according to the device types, device purposes, commonly used collaborative work contents of the user, current environments of the device, running states of the device, recently used applications of the user, and the like.

[0326] In some embodiments, the super terminal can recommend scenario-based services according to the device types, product positioning, device purposes, and the like. For example, the combination of a mobile phone and a smart watch is commonly used for counting exercise, so the super terminal system can classify the combination of the mobile phone and the smart watch into a sports health scenario based on the device types, device purposes, and commonly used collaborative work contents of the combination of the mobile phone and the smart watch. The scenario-based service options that can be provided in the sports health scenario include outdoor running, indoor running, heart health, blood vessel health, and the like.

[0327] In some embodiments, when listing the scenario-based service options, the probability of each scenario-based service option being selected can be calculated according to the recent use frequency of the user or the environment in which the current device is located, and the scenario-based service options with a large probability of being selected are displayed preferentially in the front in the scenario-based service option area.

[0328] The super terminal system can display the service options with high frequency of use in front according to the frequency of use of the service options by the user in a certain time. The certain time can be all historical time or a certain time period, such as the last three months. The service options with higher frequency of use are displayed in front.

[0329] In some other embodiments, the core device or the associated device can detect the environment where the current device is located, and sort the service options according to the environment. If the core device or the associated device detects that the user is located outdoors, the service options related to outdoor are recommended first, such as outdoor running. If the core device or the associated device detects that the user is located indoors, the service options related to indoor are recommended first, such as indoor running. The embodiments do not limit the way of detecting the environment where the user is located. For example, the current positioning information of the device can be used to determine the environment; or a camera is used to collect a photo of the current environment, or ultrasonic waves, infrared rays, etc. are used to collect the depth or size of the current space to determine the environment where the user is located.

[0330] In some other embodiments, the super terminal system can also determine the environment or the scene where the device is located according to the application recently opened or used by the user, or the currently running application, and then recommend the service options. For example, in the case that a mobile phone and a smart TV form a super terminal, if it is detected that the currently running application of the mobile phone is an office demonstration application, the multi-screen collaboration service option is recommended first; or if it is detected that the currently running application of the mobile phone is a video application, the home theater service option is recommended first; or if it is detected that the user browses a game related webpage, the game entertainment related service option is recommended first.

[0331] In some other embodiments, the super terminal system can also recommend the service options according to the running state of the device. For example, in the case that a mobile phone and a smart TV form a super terminal, if it is detected that the smart TV is in a bright screen state, the screen mirroring service is recommended first; or if it is detected that the smart TV is in an off screen state, the voice casting service is recommended first.

[0332] In some other embodiments, the super terminal system can also recommend the service options according to the atomized services that can be provided by the device.

[0333] In some embodiments, the user can customize and add the service options, such as setting a name for a certain use scene, and associating the scene with other application services or system services. The newly added service options are displayed in the super terminal interface together with other service options.

[0334] The above embodiments do not limit other embodiments, and the super terminal system can comprehensively consider one or more of the above factors to reasonably recommend the service options.

[0335] The super terminal system can automatically set the atomized service combination in the scene based on the current device combination, the queried atomized services provided by each device, the user's actual selected scenario service, the user's commonly used service combination, the user's set default service combination, the most suitable service combination for the scene based on life cognition, or the service combination with the most frequent use in the scene obtained from the server, thereby saving the user's tedious selection operation.

[0336] For example, after the mobile phone and the smart television form a super terminal, if the user selects the scenario service as cross-screen conference, the atomized service combination is automatically set as: the audio output function is provided by the smart television, the audio input (microphone) function is provided by the mobile phone, the camera function is provided by the smart television, and the display function is provided by the smart television. For another example, after the mobile phone, the smart watch, the earphone, and the treadmill form a super terminal, if the user selects the scenario service as indoor running, the atomized service combination is automatically set as: the audio output function is provided by the earphone, the audio input (microphone) function is provided by the earphone, the camera function is provided by the mobile phone, and the display function is provided by the treadmill.

[0337] In some embodiments, the super terminal system can analyze the optimal atomized service combination according to the device characteristics of different device types. For example, the smart television has a large screen, and the user's visual viewing experience is better, and it is more suitable for watching videos, so when the smart television and the mobile phone form a super terminal, the smart television is preferentially set to provide the display function. The smart speaker has strong audio output capability, and the sound effect is better, and it is more suitable for playing audio, so when the smart speaker and the mobile phone form a super terminal, the smart speaker is preferentially set to provide the audio output function. The earphone is small in size, easy to carry, does not need to be played out, does not disturb the surrounding people, and can also have a noise reduction effect to shield the external environmental noise, and is suitable for listening to audio in public places, such as making a call or listening to music in public places, so when the earphone and the mobile phone form a super terminal, the earphone is preferentially set to provide the audio output function. The smart watch and the smart bracelet can more accurately measure the user's motion data or heart rate and other health information than the mobile phone, so when the smart watch or the smart bracelet and the mobile phone form a super terminal, the motion data obtained by the smart watch or the smart bracelet is preferentially set to be used. The notebook computer has strong processing capability, and is more suitable for office work, so when the notebook computer and the mobile phone form a super terminal, the notebook computer is preferentially set to provide the display function, the keyboard input function, and the like. The device types and device characteristics are not described here again, and the above examples do not limit other embodiments.

[0338] In some embodiments, the super terminal supports user personalization setting the atomized service combination, the user can manually modify the atomized service combination according to personal preferences or actual conditions, and adjust a certain atomized service to be provided by other devices. For example, in a cross-screen conference, if the surrounding environment is relatively noisy, the user can change the audio output function from being provided by the smart TV to being provided by the earphone.

[0339] In the embodiments provided in the present application, the super terminal system can also analyze the current scene based on the types of the devices currently constituting the super terminal system, list other devices suitable for the current scene to the user to help the user obtain better scene experience, or infer more expandable use scenarios, and then list other devices to the user. If the user adopts the devices recommended by the system, more and richer use scenarios can be further expanded.

[0340] For example, after the mobile phone cooperates with the smart TV, the system can recommend wireless microphones, wireless game handles and other devices to the user, which can correspond to expand K song, game and other scenario services. For another example, after the mobile phone cooperates with the smart watch, the system can recommend running machines, heart rate bands, blood glucose meters and other devices to the user, which can correspond to expand indoor running, heart health, blood glucose health and other scenario services. After adding a recommended device, the system can also recommend new devices based on the newly constituted super terminal. The user can select to add more devices to constitute the super terminal, so that the use scenarios of the user are more abundant. After adding more associated devices, the super terminal can provide more and richer scenario services, and perfect the ecological system of the super terminal.

[0341] The functions provided by the above embodiments will be described below with reference to the accompanying drawings.

[0342] Reference is made to Figure 7 Fig. 7 shows a user interface 700, in Figure 7 In the example shown in Fig. 7, when the mobile phone cooperates with the smart watch to constitute a super terminal system, a scenario service option area 705 can be displayed below the device connection state diagram 701, and the scenario service option area 705 lists one or more scenario services recommended by the super terminal system according to the current device combination. In addition, a recommended device area 704 is also displayed, which is used to list devices that can be linked with the current super terminal system to bring richer scenarios or better experience to the user, and the user can select as needed. In the user interface 700, an atomized service entry option 706 is also displayed, and when the atomized service entry option 706 is selected, the atomized service combination can be displayed, and the user can select the providing device of the atomized service by himself / herself, which is described with reference to the embodiment shown in Figure 7 Fig. 8.

[0343] In Figure 10In the example shown in FIG. 7A, when the phone and the smart watch form a super terminal system, a plurality of scenario-based service options are displayed in the scenario-based service option area 705, such as outdoor running, indoor running, heart health, blood vessel health, and the like. The user can also swipe left and right on the scenario-based service option area 705 using a finger on the screen to view more options that are not displayed.

[0344] These scenario-based service options can be scenario-based service options that are statistically obtained by the super terminal system based on the device type, device purpose, commonly used collaborative work content of the user, current environment of the device, running state of the device, recently used application of the user, and the like. The super terminal system can also calculate the probability of selection of each scenario-based service option based on the recent use frequency of the user or the detected current environment of the device, and display scenario-based service options with a high probability of selection in the scenario-based service option area 705.

[0345] For example, the phone and the smart watch are commonly used to count exercise, and the super terminal system can classify the combination of the phone and the smart watch as a sports health scenario based on the device type, device purpose, and commonly used collaborative work content of the combination of the phone and the smart watch. The scenario-based service options that can be provided in the sports health scenario include outdoor running, indoor running, heart health, blood vessel health, and the like, which are displayed in the scenario-based service option area 705.

[0346] The super terminal system can display the options with a high use rate in the front based on the use frequency of each scenario-based service option of the user within a certain period of time. The certain period of time can be all historical time or a certain time period, such as the last three months. The higher the historical use rate of the scenario-based service option, the more it is displayed in the front. For example, it is obtained that the most commonly used scenario-based service option of the user is outdoor running, followed by indoor running, and the like. In the scenario-based service option area 705, the first displayed scenario-based service option is outdoor running, the second displayed scenario-based service option is indoor running, and the like.

[0347] In other embodiments, the phone or the smart watch can detect the environment in which the device is currently located, and sort the scenario-based service options based on the environment. If the phone or the smart watch detects that the user is located outdoors, the outdoor running service option is preferentially recommended. If the phone or the smart watch detects that the user is located indoors, the indoor running service option is preferentially recommended. The present embodiment does not limit the way in which the environment in which the user is located is detected. For example, the current positioning information of the device can be used to determine whether the user is located indoors or outdoors; or a camera can be used to capture a photo of the current environment, or ultrasonic waves, infrared rays, and the like can be used to capture the spatial depth or size of the current environment to determine whether the user is located indoors or outdoors.

[0348] In some embodiments, the super terminal system can also determine the environment or scenario the device is in according to the application the user recently opened or used, or the application currently running, and then recommend the scenario-based service option. For example, before the user opens the super terminal interface to connect the smart watch and the mobile phone, the application the user is running is a sports health application, and the service the user is using is the outdoor running service in the sports health application. Therefore, when the user connects the smart watch and the mobile phone to form the super terminal system, the first scenario-based service option displayed in the scenario-based service option area 705 is the outdoor running service.

[0349] The above examples do not limit other embodiments. The super terminal system can comprehensively consider one or more of the above factors to reasonably recommend the scenario-based service option.

[0350] In some embodiments, when the devices are connected to form the super terminal, a certain scenario-based service can be automatically selected as the default service. For example, the scenario-based service option ranked first in the scenario-based service option area 705 is set as the default service. When the devices are connected to form the super terminal, the devices are configured according to the atomic service combination corresponding to the scenario-based service by default. If the user wants to select other scenario-based services, the user can select other scenario-based service options. Setting the default scenario-based service can facilitate seamless conversion of services and can save user operations to a certain extent. For example, when the earphone is connected to the mobile phone, the default service is set to provide audio output and audio input functions by the earphone. Therefore, at the moment when the earphone and the mobile phone are successfully connected, the audio output and audio input functions are immediately switched to the earphone, and the user does not need to perform additional operations, thereby saving cumbersome operations. The default service setting can be a default service selected by the user, including a scenario-based service and / or an atomic service. The default service setting can also be a service frequently used by the user, which is set as the default service by the system, and the like. The present embodiment does not limit this.

[0351] In other embodiments, when devices are connected to form a super terminal, a scenario-based service can be listed without initially selecting one. The user then selects a scenario-based service, and the service or corresponding atomic service is switched to the desired state. This approach fully considers the user's intentions and avoids automatic switching to the wrong service, which could cause inconvenience or frustration. For example, when connecting a phone to a PC, if the user intends to transfer files, the default service would be screen mirroring, projecting the phone screen onto the PC. This would require the user to switch services, especially if the PC is currently being used for other purposes, such as presenting a document. Screen mirroring could interfere with the presentation, causing inconvenience. Alternatively, if the user has private content on their phone that they don't want displayed on the PC, directly switching to screen mirroring would be problematic. Therefore, when a phone is connected to a PC, scenario-based service options can be displayed on the super terminal interface. The user can first select the desired scenario-based service option and / or atomic service combination before switching services.

[0352] This explains that when two or more devices form a super terminal and the user selects a contextualized service, these devices can have the capability or state to perform interactive functions, but this does not limit the immediate execution of interactive functions between the two or more devices. For example, if a user triggers the audio casting function between a mobile phone and a smart speaker by dragging and dropping a device icon, and if the mobile phone does not have an audio playback task or its audio playback task is paused at this time, the mobile phone will not immediately send audio data to the smart speaker. However, if the mobile phone has an audio playback task, it will send the audio data to be played to the smart speaker for playback.

[0353] exist Figure 7 In the user interface 700, the recommended device area 704 displays icons for treadmills and exercise bikes, prompting users to add treadmills or exercise bikes to the super terminal for a better user experience. Figure 7 In the example shown, the super terminal system can analyze the user's usage scenario as a sports and health scenario based on the fact that the current devices constituting the super terminal are mobile phones and smartwatches. The recommended device area 704 can display recommended sports equipment or health monitoring equipment, such as treadmills, exercise bikes, blood glucose meters, heart rate monitors, etc.

[0354] When the user selects (clicks or drags) the device icon of the recommended device area 704, if the device is the associated device detected by the current core device, i.e., the device displayed in the device connection state diagram 701, the device can initiate connection to join the super terminal. When the user selects (clicks or drags) the device icon of the recommended device area 704, if the device is not displayed in the device connection state diagram 701, a page related to the device product can be jumped to, for example, to introduce the functions, use experience, how to use when forming a super terminal with other devices, and what experience is extended, purchase link, purchase channel, etc. of the recommended device to the user.

[0355] Joining these recommended devices can provide the user with better scene experience or expand more use scenarios. After joining a recommended device, the system can also recommend new devices according to the newly formed super terminal. The user can select to join more devices to form a super terminal, so that the user use scenarios are more abundant. For example, a mobile phone and a smart watch are connected to form a super terminal, and the system recommends devices including a treadmill. The user selects the treadmill to join the super terminal, and the system recommends devices including earphones, and recommends the user to listen to music while running. With the addition of more devices, a more perfect super terminal ecosystem can be formed.

[0356] In some embodiments, the mobile phone can display prompt information of recommended devices on the user interface, for example, after detecting that the mobile phone cooperates with the smart TV, the mobile phone finds that the sound box is also around, and then displays the prompt information such as "Recommended sound box Sound X to join the super terminal, do you allow it?" and gives options "Yes" and "No". If the user selects "Yes", the sound box Sound X is connected to the super terminal, and if the user selects "No", no other operation is performed. Or after giving the prompt information, only the option "Yes" is displayed. If the user selects "Yes", the sound box Sound X is connected to the super terminal, and if the user does not operate for more than a certain time, such as two seconds, the prompt information is automatically canceled and no other operation is performed.

[0357] In the embodiments of the present application, the services or functions provided by the super terminal across devices include but are not limited to system-level services or functions and application-level services or functions. Here, for example, the system-level services or functions can include but are not limited to voice projection and screen projection, and the application-level services or functions include but are not limited to application relay and content sharing.

[0358] Among them, the voice projection refers to sending the audio stream played on one electronic device to another electronic device for playing. For example, the voice projection of the mobile phone and the smart sound box, the voice projection of the mobile phone and the smart screen in the off-screen state, the screen projection of the mobile phone and the car machine, etc.

[0359] The screen projection includes stream projection (also referred to as heterogeneous projection) based on digital living network alliance (DLNA) technology, screen mirroring (also referred to as homogeneous projection) based on Miracast technology, and projection based on other technologies (for example, based on AirPlay). For example, projection between a mobile phone and a smart TV / PC / tablet. If a user wants to project a video or audio played on the mobile phone onto other electronic devices, stream projection can be performed. Then, the mobile phone can send a play address of the video or audio to a certain electronic device (such as a PC), and the PC plays the corresponding video or audio according to the received play address. Alternatively, the mobile phone sends video data or audio data played by itself to the PC, and the PC can directly play according to the received video data or audio data. In the projection process, the interface displayed on the mobile phone can be different from the content displayed on the PC. If the user wants to project the interface displayed on the mobile phone screen onto other electronic devices, screen mirroring can be performed. Then, the mobile phone sends the mobile phone interface data to other electronic devices (such as a PC), and the PC displays the interface of the mobile phone. In the projection process, the interface displayed on the mobile phone is the same as the mobile phone interface displayed on the PC.

[0360] Application relay refers to that one electronic device can relay an application running on another electronic device.

[0361] Content sharing includes sharing files / photos / videos / audio / texts and the like between different electronic devices. For example, application relay between a mobile phone and a smart TV / PC / tablet. The relayed application is, for example, a smooth application (a kind of video call application), a video application, a music application, and a document application, and the like.

[0362] As can be seen from the above, Figure 7 The example user interface 700 shown can be a unified entrance provided by a core device mobile phone for interaction with multiple associated devices, and a user can trigger a super terminal composed of two or more electronic devices in the user interface 700. Moreover, the super terminal can determine a use scenario according to the device type, device characteristics, product positioning, commonly used functions, device state, currently running application, and current environment of each device, and provide one or more scenario-based service options, atomized service combination options, or interaction function options based on the use scenario. The user can quickly select scenario-based services and / or atomized services according to personal needs without tedious and scattered settings, thereby solving the problem that the setting entrances of interaction functions between electronic devices are scattered, the settings are tedious, and rich use scenarios cannot be listed for users.

[0363] The embodiment of the present application provides the available scenario-based service and / or atomized service in combination with the selected device of the user, lists detailed and rich use scenarios for the user, guides the user to fully explore more application scenarios by using the function of the super terminal, and realizes the effect of recommending the scenario-based service according to the device combination.

[0364] Reference Figure 7 When the mobile phone detects that the user clicks the outdoor running option 707 listed in the scenario-based service option area 705 in the user interface 700, the mobile phone can jump to the outdoor running related page, as shown in a user interface 800. The outdoor running option 707 can be highlighted when being selected.

[0365] The user interface 800 is a sports health application page, and the sub-options of the outdoor running sub-option 801, the indoor running sub-option, the heart health sub-option and the blood vessel health sub-option can be seen displayed below the sports health title bar. Since the user selects the outdoor running option 707 in the user interface 700, the outdoor running sub-option 801 corresponds to the page displayed by the current jump to the user interface 800, for example, the map diagram, the positioning icon of the user, the motion route of the user, the statistical motion amount of the user and the like can be displayed.

[0366] Correspondingly, the smart watch displays the outdoor running related interface, and the smart watch user interface 802 is referred to. The smart watch user interface 802 can be inconsistent with the mobile phone outdoor running related user interface 800, and the smart watch and the mobile phone can share the motion data. The smart watch user interface 802 can display the motion data such as the pace detected by the smart watch, the motion distance, the heart rate and the motion time.

[0367] Figure 8 It is shown that the scenario-based service option and the recommended device can change accordingly with different device combinations, and the super terminal can provide more suitable scenario-based service options and recommended devices according to different device combinations.

[0368] Reference Figure 9 As shown in the user interface 900, in the user interface 900, the device connection state diagram indicates that the mobile phone is connected with the smart television Vision to form a super terminal, and the mobile phone icon and the smart television Vision icon are shown to be adsorbed together in the indication area 901, and it is indicated below the super terminal title that “has been coordinated with Vision”.

[0369] In the user interface 900, the scenario service option area 903 includes multi-screen collaboration, screen extension, cross-screen conference, motion sensing game, etc. The recommended device area 902 includes device icons of sound box, notebook computer, game controller. The scenario service options and the recommended device options are based on the devices included in the super terminal being a mobile phone and a smart TV Vision.

[0370] If the user selects the sound box icon in the recommended device area 902 to join the super terminal, or drags the sound box Sound X icon in the device connection state diagram to be adsorbed around the mobile phone icon, the user interface 910 can be displayed.

[0371] In the user interface 910, the device connection state diagram indicates that the mobile phone is connected with the smart TV Vision and the sound box Sound X to form a super terminal. Referring to the indication area 911, the mobile phone icon, the smart TV Vision icon, and the sound box Sound X icon are displayed to be adsorbed together, and the explanation "cooperated with Vision and Sound X" is displayed below the super terminal title.

[0372] In the user interface 910, the scenario service option area 913 includes home theater, motion sensing game, KTV, cross-screen conference, etc. The recommended device area 912 includes device icons of notebook computer and game controller. The scenario service options and the recommended device options are based on the devices included in the super terminal being a mobile phone, a smart TV Vision, and a sound box Sound X. In this example, compared with the combination of a mobile phone and a smart TV Vision, the super terminal system can infer that the combination of a mobile phone, a smart TV Vision, and a sound box Sound X is more inclined to entertainment, and therefore, the scenario services inclined to entertainment are preferentially recommended.

[0373] If the user selects the atomized service entry option in the user interface 910, one or more atomized service combinations can be displayed, such as Figure 9 As shown in the user interface 1000 and the user interface 1010, the user can select different atomized services provided by different devices.

[0374] Referring to Figure 10 , the service setting window 1001 is displayed in the user interface 1000, and the service setting window 1001 displays a plurality of atomized service options, such as picture display, sound output, sound input, camera, etc. as shown in the user interface 1000. Among them, the picture display service / function is provided by the smart TV Vision, the sound output service / function is provided by the sound box Sound X, the sound input service / function is provided by the mobile phone HuaWei P50, the camera service / function is provided by the mobile phone HuaWei P50, etc.

[0375] In this example, the option bar 1002 corresponding to the sound output includes the sound output service / function identifier, as well as the icon and name of the device smart speaker Sound X currently providing the sound output service / function.

[0376] In some embodiments, the super terminal supports user personalization setting of the atomized service combination, and the user can manually modify the atomized service combination according to personal preferences or actual conditions, and adjust a certain atomized service to be provided by another device. For example, the user can select to switch the service of playing audio provided by the mobile phone to the service of playing audio provided by the smart speaker.

[0377] If the user wants to replace the device providing the service, the user can click the control 1003, and the mobile phone can display a user interface 1010.

[0378] As can be seen, in the service setting window 1011 in the user interface 1010, the option bar 1012 corresponding to the sound output is expanded to display multiple device options, i.e., the mobile phone HuaWei P50 and the smart television Vision and the smart speaker Sound X in the super terminal. The user can click the corresponding selection control 1013 on the right side of the device name to confirm that the device provides the sound output service / function.

[0379] In some embodiments, the super terminal system can analyze the optimal atomized service combination according to the device characteristics of different device types, and assign different devices to carry different services. For example, in this example, the smart television, the smart speaker, and the mobile phone form a super terminal, the smart television has a large screen, and the user's visual viewing experience is better, so it is more suitable for watching videos, and therefore the smart television is preferentially set to provide the display function. The smart speaker has strong audio output capability and good loudspeaker effect, and is more suitable for playing audio, so the smart speaker is preferentially set to provide the audio output function. The mobile phone is light and easy to hold by the user, and the microphone is convenient to receive sound, so the mobile phone is preferentially set to provide the audio input and camera shooting functions.

[0380] In other embodiments, the atomized service combination in this scenario can also be automatically set based on the user's commonly used service combination, or the user's set default service combination, or the service combination most suitable for this scenario based on life cognition, or the service combination with the most frequent use in this scenario obtained from the server, etc., so as to save the user's tedious selection operation.

[0381] In some embodiments, the service setting window 1011 can also be provided with a blank option, i.e., the item of the atomized service is not provided by any device.

[0382] In some embodiments, the mobile phone interface may display corresponding prompts to remind the user of the device that provides the best atomic service, such as the current speaker being the best device for playing audio.

[0383] In some embodiments, once the user has finished setting up, they can scroll down the service settings window and return to the previous screen.

[0384] Figure 10 , Figure 11 , Figure 12 , Figure 13A , Figure 13B , Figure 13C The diagram shows the interaction methods and schematic interfaces for selecting other scenario-based service options.

[0385] like Figure 13D The user interface 1100 shown can display scenario-based service options as floating controls. Different scenario-based service options correspond to different areas. This interaction method is more interesting, more intuitive, the interface is simpler, and the corresponding user operation is more convenient.

[0386] In user interface 1100, when the user presses and holds the smartwatch icon, a circular floating area 1101 can be displayed on the super terminal interface, centered on the phone. The circular floating area 1101 can be divided into multiple areas, each corresponding to a scenario-based service option. For example, in user interface 1100, the circular floating area 1101 is divided into four areas, corresponding to the scenario-based service options: outdoor running, indoor running, heart health, and vascular health.

[0387] If a user wants to select a specific contextual service option, such as the outdoor running service option, the user can drag the smartwatch icon to the area corresponding to the outdoor running service option, and then drag it along the direction of the inner edge of the area, close to the phone icon. The user's finger movement trajectory can be referenced from the illustrated trajectory in user interface 1100. When the smartwatch icon reaches the snapping area, the user releases their finger, the corresponding outdoor running service is activated, and the phone jumps to display the outdoor running interface shown in user interface 800. Of course, if there is only one contextual service option, the circular floating area 1101 does not need to be divided into blocks; the same contextual service is selected regardless of which direction the user snaps their finger close to the phone icon. The selected contextual service option can be highlighted, such as... Figure 11 The area corresponding to the outdoor running service option is highlighted.

[0388] Figure 11 One of the operations is equivalent to Figure 11 and Figure 6 Two operations, namely Figure 8The user operation in the method includes associating the associated device to the core device and selecting the scene service, and the user operation efficiency is improved.

[0389] It should be noted that the scene service options in the floating area can also have a recommended order, and the embodiment does not limit the display manner of the recommended order of the scene service options. For example, four scene service options correspond to four areas, which are referred to as area A, area B, area C and area D in the first quadrant, the second quadrant, the third quadrant and the fourth quadrant respectively. The recommended order of the scene service options is outdoor running, indoor running, heart health and blood vessel health, which correspond to area A, area B, area C and area D respectively; or area B, area A, area C and area D; or area B, area C, area A and area D; or area B, area A, area D and area C; or area A, area B, area D and area C; or area B, area C, area D and area A; or area A, area D, area B and area C; or area A, area D, area C and area B; and the like. The embodiment does not limit the specific setting, which can be set according to the actual situation, such as the operation habit of the user, the frequently used area and the like.

[0390] Figure 11 Another user interaction operation is shown. Referring to the user interface 1200, when the user presses the smart watch icon, a circular floating area 1201 can be displayed on the super terminal interface with the mobile phone as the center. The circular floating area 1201 can be divided into multiple areas, and each area corresponds to a scene service option. For example, in the user interface 1200, the circular floating area 1201 is divided into four areas, which correspond to the scene service options of outdoor running, indoor running, heart health and blood vessel health.

[0391] If the user wants to select a certain scene service option, such as the outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option, and then release the finger, which means that the outdoor running service option is selected, and the service corresponding to the outdoor running is started, and the mobile phone jumps to display the interface of the outdoor running shown in the user interface 800. The user's finger movement trajectory can be referred to the trajectory in the user interface 1200. Of course, if there is only one scene service option, the circular floating area 1201 does not need to be divided, and the user drags the device icon to any position in the circular floating area 1201 and releases the icon, which means that the same scene service is selected. The selected scene service option can be highlighted, such as Figure 12 The area corresponding to the outdoor running service option in the user interface 1200 is highlighted. Similarly, the embodiment does not limit the display manner of the recommended order of the scene service options.

[0392] Figure 12 The interaction method shown is compared to Figure 12 The interactive method shown makes it easier for users to operate and reduces the probability of accidental selection.

[0393] Figure 11 , Figure 11 The circular floating area shown is only a schematic diagram. In other embodiments, the area representing the contextualized service options may also have other representations.

[0394] refer to Figure 12 The user interface 1300 shown has a square floating area for scenario-based service options. The user interface 1300 displays four areas centered on the phone, with the square floating area 1301 divided into four sections, each corresponding to a scenario-based service option: Outdoor Running, Indoor Running, Heart Health, and Vascular Health. If a user wants to select the Outdoor Running service, they can... Figure 13A The interaction method shown involves dragging the smartwatch icon to the area corresponding to the outdoor running service option, then dragging it along the direction of the inner edge of that area, close to the phone icon. When the smartwatch icon reaches the snapping area, the user releases their finger, the outdoor running service is activated, and the phone displays the outdoor running interface shown in user interface 800. Alternatively, it can be done as follows... Figure 11 The interaction method shown involves the user dragging the smartwatch icon to the area corresponding to the outdoor running service option, then releasing their finger to select the outdoor running service option. The corresponding service is then activated, and the phone displays the outdoor running interface shown in user interface 800. The selected contextualized service option can be highlighted, such as... Figure 12 The area corresponding to the outdoor running service option is highlighted. Similarly, this embodiment does not restrict the display method of the recommended order of scenario-based service options.

[0395] In some examples, such as Figure 13A The user interface 1310 shown is compared to Figure 13B Instead of displaying the central phone icon, a circular floating area (1311) appears after the user selects the smartwatch icon, containing only multiple contextual service options. For example... Figure 12The circular suspension area 1311 shown is divided into four areas, which correspond to the scenario-based service options of outdoor running, indoor running, heart health, and blood vessel health. If the user wants to select the outdoor running service, the user can drag the selected smart watch icon to the area corresponding to the outdoor running service option, which is highlighted, and then release the finger, indicating that the outdoor running service option is selected, and the service corresponding to the outdoor running is started. The phone jumps to display the interface of the outdoor running shown in the user interface 800. Similarly, the display manner of the recommended order of the scenario-based service options is not limited in this embodiment.

[0396] Figure 13B The user interface 1320 shows another display style. After the user selects the smart watch icon, the phone displays a circular suspension area 1321, which includes multiple circular ring areas, each of which corresponds to a scenario-based service option. Figure 13C The circular suspension area 1321 shown has four circular ring areas, which correspond to the scenario-based service options of outdoor running, indoor running, heart health, and blood vessel health. If the user wants to select the outdoor running service, the user can drag the selected smart watch icon to the area corresponding to the outdoor running service option, which is highlighted, and then release the finger, indicating that the outdoor running service option is selected, and the service corresponding to the outdoor running is started. The phone jumps to display the interface of the outdoor running shown in the user interface 800. Similarly, the display manner of the recommended order of the scenario-based service options is not limited in this embodiment. In an implementation, the more preferred scenario-based service options are displayed in the inner ring. Here, the circular ring with a smaller diameter is referred to as the inner ring relative to the circular ring with a larger diameter, or the circular ring closer to the center is referred to as the inner ring. For example, the outdoor running service option is the first recommended scenario-based service option, which can be displayed in the innermost circular ring. The indoor running service option is the second recommended scenario-based service option, which is displayed in the second inner circular ring. The heart health service option is the third recommended scenario-based service option, which is displayed in the third inner circular ring. The blood vessel health service option is the fourth recommended scenario-based service option, which is displayed in the outermost circular ring.

[0397] Figure 13C The user interface 1330 shows another display style. After the user selects the smart watch icon, the phone displays a fan-shaped suspension area 1331, which includes multiple fan ring areas, each of which corresponds to a scenario-based service option. Figure 13DThe illustrated fan-shaped suspension area 1331 has four fan ring areas, which respectively correspond to the scenario-based service options: outdoor running, indoor running, heart health, and blood vessel health. If the user wants to select the outdoor running service, the user can drag the selected smart watch icon to the area corresponding to the outdoor running service option, the area corresponding to the outdoor running service option is highlighted, and then the user releases the finger, which means that the outdoor running service option is selected, the outdoor running service is started, and the phone displays the interface of the outdoor running service as shown in the user interface 800.

[0398] In Figure 13D In the illustrated example, the fan-shaped suspension area 1331 is a quarter circle fan-shaped area with the center of the circle as the lower right corner, which is reasonable for the user to control the phone with the right hand for the convenience of the user. Of course, it is not limited to Figure 13D In the illustrated example, the fan-shaped suspension area 1331 can also be a quarter circle fan-shaped area with the center of the circle as the lower left corner, which is convenient for left-handed users to control. The phone can switch different display control styles according to the user's setting of the commonly used hand, right or left hand, or the phone can determine whether the user is currently using the left hand or the right hand when the user controls the phone, and then confirm the display style of the suspension control, etc.

[0399] Similarly, the embodiment does not limit the display method of the recommended order of the scenario-based service options. In one implementation, as shown in the user interface 1330, considering that the user usually holds the phone with the right hand in the lower half of the phone, the scenario-based service options that are more preferentially recommended are displayed in the inner ring, here the fan ring closer to the center of the circle is called the inner ring, or the fan ring with a smaller radius is called the inner ring. For example, the outdoor running service option is the first recommended scenario-based service option, which can be displayed in the innermost ring. The indoor running service option is the second recommended scenario-based service option, which is displayed in the second inner ring. The heart health service option is the third recommended scenario-based service option, which is displayed in the third inner ring. The blood vessel health service option is the fourth recommended scenario-based service option, which is displayed in the outermost ring.

[0400] The above embodiments are only examples, and the embodiments of the present application do not limit the form of selecting scenario-based service options and atomic service options. Technical solutions with the same inventive concept are within the protection scope of the present application.

[0401] In Figure 13D , Figure 11 , Figure 12 , Figure 13A , Figure 13B , Figure 13CIn the embodiment shown, after the user selects the outdoor running service option, the corresponding service of outdoor running is started, and the phone jumps to display the interface of the outdoor running service shown in user interface 800. When the user selects to return to the super terminal page, see Figure 13D As shown in user interface 1400, the device connection state diagram area in the super terminal page displays that the smart watch icon is attached around the phone icon, indicating that the smart watch has established a connection with the phone and can work cooperatively.

[0402] In some embodiments, when multiple devices form a super terminal and start to work cooperatively in a scenario service, the orientation information of the interaction in the interface can also be used as an input parameter, and different input of the orientation information can correspond to different feedback. For example, when the phone and the associated device such as a PAD / PC / smart TV are connected to start a multi-screen cooperative service, the user drags the associated device icon to the left or right of the phone icon or the middle area to attach and adhere, which will produce different effects. For example, if the user drags the associated device icon to the left of the phone icon to attach and adhere, the phone screen will be projected on the right side of the associated device screen; if the user drags the associated device icon to the right of the phone icon to attach and adhere, the phone screen will be projected on the left side of the associated device screen; if the user drags the associated device icon to the middle of the phone icon to attach and adhere, the phone screen will be projected on the associated device screen in full screen.

[0403] Of course, in some embodiments, the device icons of the attaching and adhering parties are exchanged in the dragging direction, and the services or interactive functions performed are the same. For example, the user can also hold the phone icon and drag it to the left or right of the associated device icon or the middle area to attach and adhere, and the service feedback will be different, which is the same as the effect described in the previous embodiment, that is, if the user drags the phone icon to the right of the associated device icon to attach and adhere, the phone screen will be projected on the right side of the associated device screen; if the user drags the phone icon to the left of the associated device icon to attach and adhere, the phone screen will be projected on the left side of the associated device screen; if the user drags the phone icon to the middle of the associated device icon to attach and adhere, the phone screen will be projected on the associated device screen in full screen. In summary, the effect of the scenario service can be different according to the orientation of the connection between the core device icon and the associated device icon.

[0404] In another example, for audio devices such as sound boxes, earphones, etc., different sound channels can be associated with different orientations of adhesion. Assuming that a user has two or more sound boxes, the user drags the icon of the first sound box from the left area of the icon of the mobile phone to adhere to it, and the two are adhered together, then the first sound box can provide left sound channel audio output; the user drags the icon of the second sound box from the right area of the icon of the mobile phone to adhere to it, and the two are adhered together, then the second sound box can provide right sound channel audio output. Other multi-channel arrangements are similar.

[0405] In other embodiments, the adhesion device icon and the adhered device icon are interchanged in the dragging direction, and the service or interactive function performed thereby can be different. For example, the direction of data transmission is different, and the device as the adhesion or the adhered device indicates that the device is the sender or receiver of the data, such as the direction of data transmission being that the adhesion device sends data stream to the adhered device. For example, when sharing files, if the PC icon is held and dragged to adhere to the mobile phone icon, the corresponding service is to send the files of the PC to the mobile phone, and if the mobile phone icon is held and dragged to adhere to the PC icon, the corresponding service is to send the files of the mobile phone to the PC.

[0406] For another example, the triggered scenario service of the device icon as the adhesion or the adhered device can also be different. For example, if the PC icon is held and dragged to adhere to the mobile phone icon, the triggered service is to send the files of the PC to the mobile phone, and if the mobile phone icon is held and dragged to adhere to the PC icon, the triggered service is to project the screen of the mobile phone onto the screen of the PC.

[0407] Reference is made to the examples shown in Figure 14 , Figure 15 , Figure 16 . Figure 17 , Figure 15 , Figure 16 It is described that when the user drags the MateBook icon close to the mobile phone icon to establish a multi-screen collaboration connection, different orientations and different areas of adhesion produce different effects.

[0408] As shown in Figure 17 , in the user interface 1500 of the mobile phone, the user drags the MateBook icon from the left area 1501 of the icon of the mobile phone to adhere to it, and when the MateBook icon is in the left area 1501, the interface can display a prompt "right screen projection", prompting the user to select the option to project the screen of the mobile phone on the right side of the screen of the MateBook. When the user drags the MateBook icon to the left area 1501 and then releases the finger, the icon of the mobile phone can be adhered to the MateBook icon. It can be seen that on the screen of the MateBook, the display interface 1502 of the mobile phone is projected on the right side of the display interface of the MateBook.

[0409] like Figure 15 As shown, in the phone's user interface 1600, the user drags the MateBook laptop icon from the right side of the phone icon 1601 to the left side. When the MateBook icon is in the right side area 1601, a prompt "Left Screen Mirroring" appears on the screen, prompting the user to select this option to mirror the phone screen to the left side of the laptop screen. When the user drags the MateBook icon to the right side area 1601 and releases their finger, the phone icon and the MateBook icon snap together. It can be seen that on the laptop screen, the phone display interface 1602 is mirrored to the left side of the laptop display interface.

[0410] like Figure 16 As shown, in the phone's user interface 1700, the user drags the MateBook laptop icon to the middle area 1701 of the phone icon. When the MateBook icon is in the middle area 1701, a prompt "Full-screen casting" appears on the interface, indicating that selecting this option will allow the phone to fill the entire screen of the laptop. When the user drags the MateBook icon to the middle area 1701 and releases their finger, the phone icon can snap to or overlap with the MateBook icon. It can be seen that the phone's display interface 1702 will be fully projected onto the laptop screen.

[0411] When a core device and an associated device are connected to perform interactive functions, the interface content displayed by the core device and the associated device may be the same or different.

[0412] In some embodiments, although the core device and the associated device display different interface content, the core device and the associated device can be used together, and the displayed interface is related, rather than each displaying unrelated content.

[0413] like Figure 17 As shown in the user interface 1800 on the mobile phone, the mobile phone is connected to the smart TV, and the scenario-based service area displays service options such as multi-screen collaboration, home theater, cross-screen conferencing, and motion-sensing games. When the user selects the home theater service option 1801, the mobile phone and smart TV can display the relevant home theater service page.

[0414] like Figure 18As shown in FIG. 18, the smart TV plays a video, and the video picture is displayed in the interface 1805, the current application title "Home Theater" is displayed on the top of the video picture, and the prompt "This device has cooperated with HuaWei P50" is displayed. The weather, time, etc. The mobile phone user interface 1810 displays the remote control interface of the Home Theater application, and the title "Home Theater" is displayed in the user interface 1810. The prompt 1802 "This device has cooperated with Vision, click to remotely control Vision", the remote control operation area 1803, the switching control 1804, etc. The remote control operation area 1803 can be used by the user to control the smart TV, and can include a home control, a switch control, a volume increase control, a volume decrease control, a return control, a menu control, a voice control, a confirmation control, up, down, left and right direction controls, etc.

[0415] The switching control 1804 is used to switch the mobile phone interface between the video picture and the remote control interface. On the remote control interface, the switching control 1804 prompts "Switch to display the picture". As shown in FIG. 19, when the user clicks the switching control 1804, the mobile phone can display the video picture 1901 in the user interface 1900, and the switching control 1902. On the video playing interface of the mobile phone, the switching control 1902 prompts "Switch to the remote control interface". If the user clicks the switching control 1902, the mobile phone can return to display the remote control interface, i.e., the user interface 1810. Figure 18

[0416] In some other embodiments, after the core device cooperates with the associated device, some functions of the core device can be migrated to the associated device, and the associated device can provide the service, and the core device can hide the service to the background and run other applications in the foreground. For example, after the mobile phone cooperates with the smart TV, the video playing service is provided by the smart TV, and the user can hide the service to the background and open the chat application in the foreground to chat while watching the video.

[0417] It can be understood that the user interfaces described above are only example interfaces, which are used to assist the reader to understand the technical solutions described in the present application, and do not limit the user interfaces of other embodiments of the present application. In other embodiments, more or less user interfaces can be added or removed, or more or less controls can be added or removed, or different human-computer interaction operations can be designed, etc., so that the user interface is more in line with the user's use experience. Figure 19

[0418] In combination with the various embodiments described above, a device interaction method provided by the embodiments of the present application is introduced below. Figure 4 to Figure 19

[0419] ​​​The following method embodiments can be applied in a communication system composed of multiple devices, such as the communication system 10 described above. The communication system can also be referred to as a super terminal. The multiple electronic devices can be specifically mobile phones, tablet computers, notebook computers, personal computers (PCs), smart televisions (which can also be referred to as smart screens, large screens, etc.), or smart watches, smart bracelets, and other wearable devices, and the application does not limit the type of devices.

[0420] The core device and the associated device establish a communication connection therebetween and can communicate, transmit data or instructions through the communication connection. The communication connection can be a wired connection or a wireless connection, or a combination of several connection modes, and the embodiments are not limited.

[0421] The communication system can be a super terminal system composed of multiple devices, and the multiple devices can be specifically a core device and one or more associated devices. The system, The system, The system, HarmonyOS or other types of operating systems, and the operating systems of the terminal devices in the communication system can be the same or different, and the application does not limit this. In some embodiments, each terminal in the communication system is equipped with HarmonyOS, and the communication system can be referred to as a HarmonyOS super virtual terminal (supervirtual device) or a HarmonyOS super terminal.

[0422] In the following method embodiments, the core device / center device (such as a mobile phone) is the control center of the super terminal system, and carries out functions such as device management, service management, and service scheduling. It can be connected and cooperated with one or more associated devices (such as notebook computers, PCs, tablet computers, smart watches, smart televisions, etc.) to provide users with various scene-based services and various atomic service combinations.

[0423] In some embodiments, the super terminal application is installed on the core device or the associated device, which facilitates the user to manage the terminal devices in the super terminal system.

[0424] It can be understood that the user can interact on any device installed with the super terminal application. The instructions and data input by the user in the super terminal application on any device can be migrated to other devices in the super terminal system.

[0425] Figure 1 to Figure 19 is a flowchart of the device interaction method provided in the embodiments, and specifically includes the following steps:

[0426] S101, the first device displays a first interface, and the first interface includes a first control indicating the first device and a second control indicating the second device.

[0427] The first control is a device option indicating the first device, and the second control is a device option indicating the second device.

[0428] The first device can be the core device, and the second device can be the associated device discovered by the first device.

[0429] The first interface is the super terminal interaction interface, and the user can control and manage each device in the super terminal system through user operations on the first interface. The embodiment does not limit the form of the super terminal interaction interface. One form can refer to the super terminal interaction interface shown in Figure 20 the described embodiment.

[0430] The first control is an icon indicating the core device, such as the phone icon shown in Figure 5 . The second control is an icon indicating the associated device, such as the smart watch icon shown in Figure 5 .

[0431] Referring to Figure 5 , the device connection state diagram can be displayed in the super terminal interaction interface. The device connection state diagram has a core device icon in the center and one or more associated device icons around the core device icon.

[0432] The device connection state diagram can indicate the connection relationship, relative position relationship, signal strength relationship, etc. between the core device and the associated device.

[0433] The connection relationship can refer to the phone icon and the smart watch icon shown in Figure 5 . The phone icon and the smart watch icon are attracted together, indicating that the phone and the smart watch are connected and can work together to provide system-level services. The smart bracelet icon is separated from the phone icon, not overlapping or adhering to the phone icon, but in a separate state, indicating that the phone and the smart bracelet are not connected.

[0434] In some embodiments, the relative positions of the associated device icons and the core device icon displayed in the device connection state diagram can indicate the position relationship, such as the relative direction information and the relative distance information, of the associated device and the core device in the real space, which helps the user to quickly find and distinguish each electronic device. For example, the positions of the core device and the associated device in the real space are consistent with the positions of the core device icon and the associated device icon in the device connection state diagram. For another example, the farther the associated device is from the core device in the real space, the farther the distance between the associated device icon and the core device icon displayed in the interface is. The embodiment does not limit the specific technology for implementing this function. For example, the phone can obtain the direction of the directional information carried by the associated device based on the UWB technology and measure the distance to obtain the spatial mapping relationship of the associated device and the phone in the real space.

[0435] In some embodiments, the distance between the associated device icon and the core device icon in the interface can indicate the signal strength of the associated device detected by the core device, such as the Bluetooth signal strength or the WiFi signal strength. For example, the stronger the signal strength of the associated device detected by the core device, the closer the distance between the associated device icon and the core device icon displayed in the interface.

[0436] In S102, the first device detects the first user operation.

[0437] If the user wants the second device to form a super terminal with the first device to implement some collaborative functions, the first user operation of selecting the second device can be implemented. The specific manner of the first user operation is not limited in the embodiments. The first user operation is used for the user to associate the first device and the second device.

[0438] In an implementation manner, the first user operation can be a user operation on the second control, such as the user holding the second control and dragging the second control close to the first control, and releasing the second control when the second control reaches a specified area. In some embodiments, the specified area can be a circular area with the first control as the center and a first radius distance.

[0439] Reference Figure 7 In the embodiment shown, the user can select the smart watch icon 506 corresponding to the smart watch on the super terminal interface, and drag the smart watch icon 506 close to the mobile phone icon 505. When the distance between the smart watch icon 506 and the mobile phone icon 505 is less than or equal to a second distance (such as R2 shown in FIG. 6B), or when the smart watch icon 506 enters a specified area (such as the annular area formed by the nearest circle around the mobile phone icon, that is, the circular area with the mobile phone icon 505 as the center and a radius R), the user releases the finger, and the device connection state diagram 701 shown in FIG. 7 can be displayed. Figure 6 Figure 6 Figure 6 In the device connection state diagram 701 shown, the smart watch icon 702 is adsorbed on the edge of the mobile phone icon 505.

[0440] In some embodiments, the selected device icon can change position along with the track of the user's finger dragging on the screen. The finger movement track of the user dragging the device icon is not limited in the embodiments, and the user dragging track can be any curve. For example, when the user selects a certain associated device icon, the user drags the associated device icon on the screen for a distance or stops the associated device icon at a certain position on the screen for a second preset time length, the mobile phone can detect whether the distance between the associated device icon and the mobile phone icon is less than or equal to a certain distance, such as the second distance. If yes, the associated device icon and the mobile phone icon are adsorbed together. The associated device icon moves along with the track of the user's finger when the user's finger moves.​​

[0441] In some other embodiments, the selected device icon can also not change its position along with the trajectory of the user's finger dragging on the screen. For example, when the user selects a certain associated device icon, the user's finger slides on the screen for a distance or stays at a certain position on the screen for a second preset time length, the phone can determine whether the distance between the current position of the user's finger and the phone icon is less than or equal to a certain distance, such as a second distance, and if so, the associated device icon and the phone icon are attracted together. When the user's finger moves, the associated device icon does not move.

[0442] In another implementation, the first user operation can be that the user clicks the second control in the unconnected state, or simultaneously clicks the first control and the second control, which can trigger the connection of the second device and the first device, and on the interface, the second device icon and the first device icon are attracted together.

[0443] In another implementation, the first user operation can also be a user operation acting on the first control, such as the user holding the first control and dragging the first control close to the second control, and releasing the first control when the first control reaches a specified area, which can also achieve that the first control and the second control are attracted together.

[0444] Optionally, in another implementation, the first user operation can also be a user operation of selecting the second control, such as holding the second control, which is described in the embodiments of Figure 7 、 Figure 11 、 Figure 12 、 Figure 13A 、 Figure 13B 、 Figure 13C .

[0445] S103, in response to the first user operation, the first device displays a second interface, and the second interface displays one or more service options of a scene service that can be cooperatively provided by the first device and the second device.

[0446] Optionally, the second interface displays the first control connected with the second control. In response to the first user operation, the second interface can display that the second control and the first control are attracted together. The two device icons attracted together can mean that the edges of the two device icons are tangent, or the two device icons attracted together can also mean that the two device icons are completely or partially overlapped. The embodiment does not specially limit the form of the attraction.

[0447] The second interface in the embodiment can refer to the user interface 700 shown in Figure 13D .

[0448] In some embodiments, the second control in the second interface can be different from the representation of the second control in the first interface. For example, in Figure 7 some embodiments, the smart watch icon 702, which has connected with the phone to form a super terminal, can display a distinguished mark, so as to be distinguished from the icon of the associated device in the unconnected state. For example, the smart watch icon 702 is displayed in a dark filled form, indicating that the smart watch is in a connected state with the phone. In addition, the size of the smart watch icon 702 is smaller than that of the smart watch icon 506 in the unconnected state, the diameter of the circular icon of the smart watch icon 702 is R2, and the smart watch icon 702 is adsorbed on the edge of the phone icon 505 and is tangent to the edge of the phone icon 505.

[0449] Optionally, in another implementation, referring to the embodiments described in Figure 7 , Figure 11 , Figure 12 , Figure 13A , Figure 13B , Figure 13C , in response to the first user operation of selecting the second control, a floating control is displayed in the second interface, and the floating control has one or more areas, and each area corresponds to a scene service option. The floating control can be in the form of a plurality of circular areas, or a plurality of rectangular areas, or a plurality of sector areas, or a plurality of sector ring areas, or a plurality of circular ring areas, or a plurality of polygonal areas. In the second interface, the first control can be displayed or not displayed. The embodiments do not limit the layout of the floating control.

[0450] Referring to the user interface 1100 shown in Figure 13D or the user interface 1200 shown in Figure 11 or the embodiments shown in Figure 12 , Figure 13A , Figure 13B , Figure 13C , the second interface displays a floating control, which can be divided into one or more areas, and each area corresponds to a scene service option. Different scene service options correspond to different areas. Such an interactive mode is more interesting, more intuitive, and the interface is more concise. The embodiments do not limit the shape of the floating control, and do not limit the way of dividing areas.

[0451] For example, in the user interface 1100, when the user presses the smart watch icon, the super terminal interface can be centered on the phone to display a circular floating area 1101, and the circular floating area 1101 can be divided into a plurality of areas, and each area corresponds to a scene service option. For example, in the user interface 1100, the circular floating area 1101 is divided into four areas, and the scene service options corresponding to the four areas are outdoor running, indoor running, heart health, and blood vessel health, respectively.

[0452] The second interface displays one or more service options of a scenario service. The scenario service refers to a service based on a use scenario, and is a collection of multiple functions or services that can be provided for a specific scenario. One scenario service can be supported by one or more atomic services. The service options of the scenario service displayed on the second interface can be scenario services that can be provided by the current super terminal system, i.e., the combination of the first device and the second device, and the atomic capabilities corresponding to the scenario services have been installed and deployed on the first device or the second device.

[0453] With reference to the description in the foregoing embodiments, the scenario service refers to a functional service based on a use scenario, and is a collection of multiple functions or services that can be provided for a specific scenario. One scenario service can be supported by one or more atomic services. The service options of the scenario service displayed on the second interface can be scenario services that can be provided by the current super terminal system, i.e., the combination of the first device and the second device, and the atomic capabilities corresponding to the scenario services have been installed and deployed on the first device or the second device.

[0454] In some embodiments, a scenario service can be based on a same application, and different devices provide different atomic capabilities under the application service. The scenario service is actually provided by the same service provider, for example, Figure 13D In the example shown, the mobile phone and the smart watch both install a sports health application. After the user selects the scenario service option of outdoor running, the mobile phone and the smart watch both jump to the outdoor running service in the sports health application. The smart watch can provide a positioning service and a service of counting user sports data, and can upload the user data to an application server, which in turn synchronously sends the user data to the mobile phone. The mobile phone can provide a service of displaying the user's sports track. For the same scenario service, even if the same client is used, the interfaces displayed by different devices and the services provided can be different.

[0455] The scenario service can be a scenario service option calculated by the super terminal system based on the device type, device characteristics, product positioning, device purpose, collaborative services that can be provided by the core device and / or the associated device, the current environment of the device, the running state of the device, the recently used application of the user, the frequency of use of the scenario service by the user, the currently running application, the sorting or default option of the scenario service set by the user, and the like, and is displayed on the user interface to facilitate the user to quickly start the related service or function. The scenario service option of the scenario service with a higher probability of being selected by the user calculated by the super terminal system is more preferentially recommended and displayed.

[0456] In some embodiments, the super terminal can recommend a scenario service based on the device type, product positioning, and device purpose. For example, the combination of a mobile phone and a smart watch is commonly used to count sports, and therefore the super terminal system can classify the combination of the mobile phone and the smart watch as a sports health scenario based on the device type, device purpose, and commonly used collaborative work content of the combination of the mobile phone and the smart watch. The scenario service options that can be provided under the sports health scenario include outdoor running, indoor running, heart health, and blood vessel health.

[0457] For another example, different device types have different device features. For example, a smart TV has a large screen, and a user has a better visual experience, and is more suitable for watching videos. A smart speaker has strong audio output capability, and has a better sound effect, and is more suitable for playing audio. An earphone is small and easy to carry, and does not need to be played externally, and does not disturb people around, and can also have a noise reduction effect, shield external environmental noise, and is suitable for listening to audio in public places, such as making a call, listening to music, and the like. A smart watch and a smart bracelet can more accurately measure user motion data or heart rate and other health information than a mobile phone. A notebook computer has strong processing capability, and is more suitable for office work. Here, the device types and device features are not described again, and the above examples do not limit other embodiments.

[0458] In some embodiments, when listing the scenario service options, the probability of selection of each scenario service option can be calculated according to the frequency of recent use of different collaborative services by the user or detection of the environment in which the current device is located, and the scenario service options with a high probability of selection are displayed in the scenario service option area.

[0459] The super terminal system can display the options with a high usage rate in the front according to the frequency of use of each scenario service option by the user in a certain period of time. The certain period of time can be all historical time, or a certain time period, such as the last three months. The higher the historical usage rate of the scenario service option, the more in the front.

[0460] In other embodiments, the core device or the associated device can detect the environment in which the current device is located, and sort the scenario service options according to the environment. If the core device or the associated device detects that the user is in an outdoor environment, an outdoor related service option is preferentially recommended, such as outdoor running. If the core device or the associated device detects that the user is in an indoor environment, an indoor related service option is preferentially recommended, such as indoor running. The present embodiment does not limit the way of detecting the environment in which the user is located. For example, the current positioning information of the device can be used to determine the environment; or a camera is used to collect a photo of the current environment, or ultrasonic waves, infrared rays, and the like are used to collect the depth or size of the current space, to determine the environment in which the user is located.

[0461] In some embodiments, the super terminal system can also determine the environment or scenario where the device is located according to the application that the user recently opened or used, or the application that is currently running, and then recommend the scenario-based service option. For example, in the case that the mobile phone and the smart TV form a super terminal, if it is detected that the application currently running on the mobile phone is an office presentation application, then the multi-screen collaboration service option can be preferentially recommended; or if it is detected that the application currently running on the mobile phone is a video application, then the home theater service option can be preferentially recommended; or if it is detected that the webpage browsed by the user is a game-related webpage, then the game entertainment-related service option can be preferentially recommended, and so on.

[0462] In some other embodiments, the super terminal system can also recommend the scenario-based service according to the running state of the device. For example, in the case that the mobile phone and the smart TV form a super terminal, if it is detected that the smart TV is in a bright screen state, then the scenario-based service preferentially recommended is screen mirroring, and if it is detected that the smart TV is in an off screen state, then the scenario-based service preferentially recommended is the voice casting service.

[0463] Optionally, in some embodiments, the super terminal system can include a first database, which records one or more scenario-based services supported by one or more device combinations, the device combination including a device combination formed by the first device and its associated second device, and the first scenario-based service being a scenario-based service queried from the first database. The first database can be stored locally or on a cloud server. The scenario-based service is determined by one or more of the following parameters: device type, device characteristics, product positioning, device purpose, environment or scenario where the device is located, state where the device is located, recently running application, and so on.

[0464] Optionally, in some embodiments, a second database can be included in the super terminal system, the second database records one or more scenario services supported by one or more atomic service combinations, and the first scenario service is a scenario service queried from the second database. The second database can be stored on a local server or a cloud server. The atomic service combination includes an atomic service combination formed by atomic capabilities of the first device and the second device associated therewith. The atomic capabilities include one or more of the following: audio output capability, audio input capability, display capability, camera capability, touch input capability, keyboard and mouse input capability, etc. For example, the audio output capability can include support for single-channel or multi-channel, supported sound effects, supportable frequency response range, noise reduction capability, audio resolution capability, etc. when the device plays audio; the audio input capability can include sound collection range, noise reduction capability, etc.; the display capability can include screen size, display resolution parameters, refresh rate, color performance, etc. of the device; the camera capability can include camera type, shooting pixels, night scene shooting capability, image adjustment capability, etc. of the device; the touch input capability and the keyboard and mouse input capability indicate whether the device can support touch input or keyboard and mouse input, etc.

[0465] In the embodiment, the layout of the floating control is different, and the arrangement order of the recommended scenario services is also different.

[0466] Optionally, with reference to the embodiments shown in Figure 8 、 Figure 13C , the shape of the floating control is a circular ring or a fan ring, and the plurality of circular ring regions or fan ring regions respectively indicate a plurality of scenario services. The scenario service indicated by the circular ring region or fan ring region closer to the center is a more preferentially recommended scenario service.

[0467] Optionally, with reference to the embodiments shown in Figure 13D 、 Figure 11 、 Figure 12 、 Figure 13A , the shape of the floating control is a circle or a rectangle, the circle or the rectangle is divided into four regions, the first quadrant direction is region A, the second quadrant direction is region B, the third quadrant direction is region C, and the fourth quadrant direction is region D. The recommended order of the scenario service options is scenario service option A, scenario service option B, scenario service option C, and scenario service option D, which are respectively placed in region B, region A, region C, and region D, or in region B, region C, region A, and region D, or in region A, region D, region B, and region C, or in region A, region B, region D, and region C, or in region B, region A, region D, and region C, or in region A, region B, region C, and region D.

[0468] Of course, the scenario service options are not fixed, and the super terminal system can continuously update, expand or refine more scenario service options, and the embodiments are not limited, and the specific conditions can be changed according to the actual situation. In some embodiments, the user can customize and add scenario services, such as the user setting a name for a use scenario, and associating the scenario with other application services or system services. The newly added scenario service option is displayed in the super terminal interface together with other scenario service options.

[0469] The above embodiments do not limit other embodiments, and the super terminal system can comprehensively consider one or more of the above factors to reasonably recommend scenario service options.

[0470] Specific examples can refer to the description of the embodiments shown in Figure 13B , Figure 7 Here, the details are not repeated.

[0471] In some embodiments, when the devices are connected to form a super terminal, a certain scenario service can be automatically selected as a default service. For example, the scenario service option in the first position in the scenario service option area is set as the default service, and when the devices are connected to form a super terminal, each device is configured according to the atomic service combination corresponding to the scenario service by default. If the user wants to select other scenario services, other scenario service options can be selected. Setting a default scenario service can facilitate seamless conversion of devices and can save user operations to a certain extent. For example, when the earphone is connected to the mobile phone, the default service setting position provides audio output and audio input functions by the earphone. Therefore, at the moment when the earphone and the mobile phone are successfully connected, the audio output and audio input functions are immediately switched to the earphone, and the user does not need to perform additional operations, thereby saving cumbersome operations. The default service setting can be a default service selected by the user, including a scenario service and / or an atomic service, and the default service setting can also be a service frequently used by the user, which is set as a default service by the system, and the like. The embodiments are not limited in this regard.

[0472] In some embodiments, when the devices are connected to form a super terminal, the scenario service can be selected first, and then the scenario service or the corresponding atomized service is switched to the set state. In other embodiments, when the devices are connected to form a super terminal, the scenario service can not be selected first, and only one or more scenario service options are listed, and the user needs to select a certain scenario service before the scenario service or the corresponding atomized service is switched to the set state. The advantage of this is to fully consider the user's use intention, avoid automatic switching to the wrong service, and cause trouble or trouble to the user. For example, when a mobile phone is connected with a PC, the user's intention is to transfer files, and if the default service is screen mirroring, the mobile phone screen will be projected onto the PC. At this time, the user needs to switch the service again, especially if the PC is currently being used for other purposes, such as presenting a document, and the screen mirroring service will interfere with the display of the presentation document, causing trouble to the user, or the user's mobile phone has private content and does not want to display it on the PC. If the screen mirroring service is directly switched, it will cause trouble to the user. Therefore, when the mobile phone is connected with the PC, the scenario service options can be displayed on the super terminal interface, and the user can first select the required scenario service option and / or atomized service combination option, and then execute the service switching.

[0473] Here, it is explained that when two or more devices form a super terminal and the user selects a scenario service, the devices can have the ability or state to perform interactive functions, but it is not limited that the two or more devices immediately perform interactive functions. For example, if the user triggers the audio projection function between the mobile phone and the smart speaker by dragging the device icon. If the mobile phone does not have an audio playback task or the audio playback task of the mobile phone is paused at this time, the mobile phone will not immediately send audio data to the smart speaker. However, when the mobile phone has an audio playback task, the mobile phone will send the audio data to be played to the smart speaker for playback by the smart speaker.

[0474] In the second interface, there can also be a recommended associated device, which can also be referred to as a third device. The super terminal system can also analyze the current scenario based on the types, device characteristics, product positioning, device purposes, environments or scenarios in which the devices are located, states in which the devices are located, and recently running applications of the devices currently forming the super terminal system, list recommended devices suitable for the current scenario to the user, to help the user obtain better scenario experience; or predict more expandable use scenarios, and then list recommended devices to the user. If the user adopts the devices recommended by the system, more and richer use scenarios can be further expanded.

[0475] Optionally, the third device is a device discovered by the first device, and the first device displays service options of a second scenario service by clicking the device option of the third device. Among them, the service options of the second scenario service can add some scenario service options on the basis of the service options of the first scenario service, or the service options of the second scenario service are updated scenario services, which are different from the service options of the first scenario service.

[0476] Optionally, the third device is not the device discovered by the first device, and the first device jumps to display a fifth interface when the device option of the third device is clicked, the fifth interface can be a purchase interface of the third device, the fifth interface includes a plurality of item options, the plurality of item options come from a plurality of providing sources, and the plurality of item options include an item option indicating the third device.

[0477] For example, after the mobile phone cooperates with the smart television, the mobile phone can recommend a wireless microphone, a wireless game handle and the like to the user, and can correspondingly expand K song, game and the like scene service. For another example, after the mobile phone cooperates with the smart watch, the mobile phone can recommend a treadmill, a heart rate belt, a blood glucose meter and the like to the user, and can correspondingly expand indoor running, heart health, blood glucose health and the like scene service. After a recommended device is added, the system can also recommend a new device according to a newly formed super terminal. The user can select to add more devices to form a super terminal, so that the use scene of the user is more abundant. After more associated devices are added, the super terminal can provide more and richer scene services, and perfect the ecological system of the super terminal.

[0478] In Figure 9 In the example shown, the super terminal system can analyze that the use scene of the user is a sports health scene based on the devices currently forming the super terminal, which are the mobile phone and the smart watch, and the recommended device area 704 can display recommended sports devices or health monitoring devices, such as a treadmill, a spinning bike, a blood glucose meter, a heart rate meter and the like. When the user selects (clicks or drags) the device icon of the recommended device area 704, if the device is an associated device detected by the current core device, that is, the device displayed in the device connection state diagram 701, the device can initiate connection to join the super terminal. When the user selects (clicks or drags) the device icon of the recommended device area 704, if the device is not displayed in the device connection state diagram 701, a page related to the product of the device can be jumped to, for example, the user is introduced to the functions, use experience, use when forming a super terminal with other devices and the like experience that can be expanded of the recommended device.

[0479] S104, the first device detects a second user operation of selecting a first scene service, the first scene service instructs the first device to provide a first service and / or the second device to provide a second service.

[0480] The first scene service corresponds to a first atomic service combination, the first atomic service combination provides at least one first service for the first device and / or at least one second service for the second device. One scene service can be supported by one or more atomic services.

[0481] The atomized service refers to a minimum capability unit that can be independently run, is a concept of abstract encapsulation of a single function / capability, and can be a hardware service or a software service.

[0482] The second user operation is not limited in the embodiment. With reference to the embodiment shown in Figure 7 , the second user operation can be that the user clicks the first scenario service option.

[0483] Optionally, with reference to the embodiment described in Figure 8 , Figure 11 , Figure 12 , Figure 13A , Figure 13B , Figure 13C , the second user operation can be that the user drags the second control to the area corresponding to the first scenario service option and then releases the second control; or the second user operation can be that the user drags the second control to the area corresponding to the service option of the first scenario service and then releases the second control to be close to the first device icon. The motion trajectory of the user dragging is not limited in the embodiment.

[0484] In some embodiments, the aforementioned floating control can include a first area displaying the service option of the first scenario service. The second user operation is that the user drags the second control to the first area and then releases the second control, or the second user operation is that the user drags the second control to the first area and then releases the second control to move to a specified area in the direction close to the first control. The specified area can be a circular area with the first control as the center and a first radius distance as the radius.

[0485] For example, in the example shown in Figure 13D , if the user wants to select a certain scenario service option, such as the outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option and then release the finger to be close to the mobile phone icon in the direction of the outdoor running service option. The motion trajectory of the user's finger can be referred to the schematic trajectory in the user interface 1100. When the smart watch icon reaches the adsorption area, the user releases the finger, the service corresponding to the outdoor running is started, and the mobile phone jumps to display the interface of the outdoor running shown in the user interface 800. Of course, if there is only one scenario service option, the circular floating area 1101 does not need to be divided, and the user approaches the mobile phone icon from any direction to select the same scenario service.

[0486] For another example, in the example shown in Figure 11In the shown example, if the user wants to select a certain scenario service option, such as the outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option, and then release the finger, which means that the outdoor running service option is selected, and the service corresponding to the outdoor running service is started, and the phone jumps to display the interface of the outdoor running shown in the user interface 800. The user's finger movement trajectory can refer to the schematic trajectory in the user interface 1200. Of course, if there is only one scenario service option, the circular floating area 1201 does not need to be divided, and the user drags the device icon to any position in the circular floating area 1201 and releases the icon to select the same scenario service.

[0487] Figure 12 The interaction mode shown is more simple and less likely to be misselected than Figure 12 The interaction mode shown is more simple and less likely to be misselected than

[0488] The super terminal system can automatically set the atomic service combination in the scenario based on the current device combination, the atomic services provided by each device found by the query, the scenario service actually selected by the user, or based on the service combination commonly used by the user, or the default service combination set by the user, or the service combination most suitable for the scenario based on life cognition, or the service combination with the most frequent use in the scenario obtained from the server, etc., saving the user from the tedious selection operation.

[0489] For example, after the phone and the smart TV form a super terminal, if the scenario service selected by the user is cross-screen conference, the atomic service combination is automatically set as: the audio output function is provided by the smart TV, the audio input (microphone) function is provided by the phone, the camera function is provided by the smart TV, and the display function is provided by the smart TV. For another example, after the phone, the smart watch, the earphone, and the treadmill form a super terminal, if the scenario service selected by the user is indoor running, the atomic service combination is automatically set as: the audio output function is provided by the earphone, the audio input (microphone) function is provided by the earphone, the camera function is provided by the phone, and the display function is provided by the treadmill.

[0490] In some embodiments, the super terminal system can analyze the optimal atomized service combination according to the device features of different device types. For example, a smart TV has a large screen, and the user visual viewing experience is good, and is more suitable for watching videos. Therefore, when the smart TV and the mobile phone form a super terminal, the smart TV is preferentially set to provide the display function. The smart speaker has strong audio output capability, and the sound effect is good, and is more suitable for playing audio. Therefore, when the smart speaker and the mobile phone form a super terminal, the smart speaker is preferentially set to provide the audio output function. The earphone is small in size, easy to carry, does not need to be played out, does not disturb the surrounding people, and can also have a noise reduction effect, shielding the external environmental noise, and is suitable for listening to audio in public places, such as calling, listening to music, etc. Therefore, when the earphone and the mobile phone form a super terminal, the earphone is preferentially set to provide the audio output function. The smart watch and the smart bracelet can more accurately measure the motion data or heart rate and other health information of the user than the mobile phone. Therefore, when the smart watch or the smart bracelet and the mobile phone form a super terminal, the motion data obtained by the smart watch or the smart bracelet is preferentially set. The notebook computer has strong processing capability, and is more suitable for office work. Therefore, when the notebook computer and the mobile phone form a super terminal, the notebook computer is preferentially set to provide the display function, the keyboard input function, etc. The device types and device features are not described here, and the above examples do not limit other embodiments.

[0491] In some embodiments, the super terminal supports user personalization setting of the atomized service combination, and the user can manually modify the atomized service combination according to personal preferences or actual conditions, and adjust a certain atomized service to be provided by other devices. For example, in a cross-screen conference, if the surrounding environment is relatively noisy, the user can change the audio output function from the smart TV to the earphone.

[0492] Optionally, the first device displays the first atomized service combination on the seventh interface, the first atomized service combination is selected by the user, or the first atomized service combination is an atomized service combination determined according to one or more of the following parameters: the device type and the device feature of the first device and / or the second device, or the atomized service combination commonly used by the user, or the default atomized service combination set by the user, or the atomized service combination most suitable for the first scenario service analyzed, or the atomized service combination with the highest use frequency obtained from the server under the first scenario service.

[0493] Referring to the examples shown in the foregoing Figure 11 If the user selects the atomized service entry option in the user interface 910, one or more atomized service combinations can be displayed, such as the user interface 1000 and the user interface 1010 shown in Figure 10 The user can select different atomized services provided by different devices by himself / herself.

[0494] It can be understood that the scenario service and the atomic service are only words used in the embodiment, and the meanings represented by the words have been described in the embodiment, and the names of the words do not constitute any limitation on the embodiment. For example, in some other embodiments, the scenario service can also be referred to as a scenario function, a scenario service, a scenario service, a service function, or other names; and the atomic service can also be referred to as a meta ability, an atomic ability, an atomic service, a functional component, or other names. In the embodiment of the application, the scenario service and the atomic service are mainly described.

[0495] In S105, the first device and / or the second device runs the first scenario service, that is, the first device runs the first service, and / or the second device runs the second service.

[0496] In the embodiment, the way in which the first device and / or the second device runs the scenario service is not limited. In some embodiments, the first device can directly notify the second device of running the atomic service corresponding to the first scenario service through a communication connection. In some other embodiments, the first device can send an instruction of running the first scenario service to a cloud server, or the first device can report the instruction or a message obtained by the first device or a running state of the first device to the cloud server, and then the cloud server generates an instruction of running the first scenario service and sends the instruction to the second device. After receiving the instruction of running the first scenario service, the second device runs the atomic service indicated by the first scenario service.

[0497] The third interface displayed by the first device running the first scenario service can be different from or the same as the fourth interface displayed by the second device running the first scenario service.

[0498] Optionally, the first scenario service is a motion monitoring service, the second device is a wearable motion device, the third interface can be a picture of a motion track, and the fourth interface can be motion data of a user detected, and the motion data can include motion speed, motion distance, motion time, heart rate, and the like.

[0499] Reference Figure 10As shown in the example, when the phone detects that the user has selected the outdoor running option 707 listed in the contextualized service option area 705 of the user interface 700, the phone can jump to the outdoor running-related page, as shown in the user interface 800. User interface 800 is the sports and health application page, and you can see sub-options displayed below the sports and health title bar: outdoor running sub-option 801, indoor running sub-option, heart health sub-option, and vascular health sub-option. Since the user selected the outdoor running option 707 in user interface 700, the current jump to user interface 800 displays the page corresponding to the outdoor running sub-option 801, which may display a map, the user's location icon, the user's exercise route, and statistics on the user's exercise volume, etc.

[0500] Accordingly, the smartwatch displays an interface related to outdoor running, as shown in smartwatch user interface 802. The smartwatch user interface 802 may differ from the mobile phone's outdoor running user interface 800, allowing the smartwatch and phone to share exercise data. The smartwatch user interface 802 can display exercise data detected by the smartwatch, such as pace, distance, heart rate, and time.

[0501] Optional, see reference Figure 8 In the embodiment shown, the first scenario service is a screen projection service, the second device is a large-screen device, the fourth interface is a video or image screen, and the third interface is a control interface for controlling the display function of the video or image screen.

[0502] Optionally, the content displayed on the third and fourth interfaces can be the same. (See reference) Figure 18 In the illustrated embodiment, both the mobile phone and the television display the same video feed.

[0503] Optionally, when the user chooses to return to the HyperTerminal page, a second control can be displayed that is snapped together with the first control. Snapping together can mean that the edges of the corresponding icons of the two devices are tangent, or it can mean that the two device icons are wholly or partially overlapping. This embodiment does not impose any special limitations on the form of snapping. For example, see [link to example]. Figure 19 The user interface 1400 shown in the SuperTerminal page displays a smartwatch icon attached to a phone icon, indicating that the smartwatch has established a connection with the phone and can work together.

[0504] In some embodiments, when a user drags the associated device icon from different locations or regions to snap it toward the core device icon, the resulting effects and triggered services may be different.

[0505] In the screen projection scenario, the first device determines the position of the screen interface of the first device in the screen of the second device according to the relative position between the second control and the first control when the drag operation is released. For details, refer to the embodiments described in Figure 14 , Figure 15 , Figure 16 .

[0506] If the relative position is that the second control is on the left side of the first control, the screen interface of the first device is projected on the right side of the screen of the second device; if the relative position is that the second control is on the right side of the first control, the screen interface of the first device is projected on the left side of the screen of the second device; if the relative position is that the second control is in the middle of the first control, the screen interface of the first device is projected on the screen of the second device in full screen.

[0507] If the user wants to disconnect a certain associated device from the super terminal system or end the service, the fourth user operation can be performed in the super terminal interface when the associated device icon and the core device are in the state of being attracted, and the fourth user operation includes the operation of separating the first control from the second control. For example, the second control is selected and dragged away from the first control until it moves out of the specified area, and then the second control is released. In an example, the user can select the associated device icon and drag the associated device icon away from the core device icon. When the mobile phone detects that the user drags the associated device icon away from the core device icon by more than a third distance, the associated device icon returns to the position before being connected when the user releases the finger, the state of being attracted is released, the associated device is disconnected from the core device, and the associated device is disconnected from the super terminal system. The services provided by the device in the super terminal are also terminated. Alternatively, when the user clicks on a certain associated device in the connected state, or the user holds the associated device icon and the core device icon and drags them in the opposite direction to separate them, the associated device and the core device can be disconnected, and the associated device icon and the core device icon are separated on the interface, and the associated device icon returns to the position in the unconnected state. The embodiments do not limit the conditions or user operations for triggering attraction and releasing attraction.

[0508] In some embodiments, a one-key disconnection control can also be provided on the super terminal interface. Clicking the one-key disconnection control can disconnect all associated devices from the core device, restore the state of the core device being independent and having no associated device connected, and terminate the services provided by the associated devices.

[0509] It can be understood that, not limited to the connection between the associated device and the core device to form a super terminal, in other embodiments, the user can also connect a certain associated device and another associated device in the super terminal interface to form a super terminal. That is, the user can connect any two or more devices, and the devices having a connection relationship are not limited to be associated devices or core devices, such as the associated device smart sound box and the smart television which can be directly connected to form a super terminal, and various functions, schematic interfaces or user operations thereof can refer to the description in the foregoing embodiments, which will not be described here.

[0510] In some embodiments, the first device detects a third user operation acting on the second interface, the third user operation being used to associate a fourth device and the first device, the third user operation including a third control operation of selecting a third control indicating the fourth device. In response to the third user operation, the first device displays a sixth interface, the first control, the second control and the third control being displayed to be adsorbed together in the sixth interface, and the sixth interface further displays a third scene service option, the third scene service option being determined according to the first device, the second device and the fourth device, and the third scene service option being different from the first scene service option. For example, the first device is a smart television, the second device is a smart sound box, and the fourth device is a smart air conditioner. The first scene service option is a scene service option of the smart television, the smart sound box and the smart air conditioner being connected to form a super terminal, and the third scene service option is a scene service option of the smart television, the smart sound box and the smart air conditioner being connected to form a super terminal. Figure 17 The embodiments shown will not be described here.

[0511] As can be seen from the above, the embodiments of the present application provide a unified entry for interaction between the core device and the plurality of associated devices, that is, the super terminal interface, and the user can trigger any two or more electronic devices to form a super terminal in the super terminal interface. Moreover, the super terminal can determine a use scenario according to the device type, device characteristics, product positioning, commonly used functions, device state, current running application, current device environment and the like of each device, and provide one or more scene service options, atomized service combination options or interaction function options based on the use scenario. The user can quickly select the scene service and / or the atomized service according to personal needs without tedious and scattered settings, thereby solving the problems of scattered setting entry of the interaction function between electronic devices, tedious setting and inability to list rich use scenarios for the user.

[0512] The embodiments of the present application provide the available scene service and / or atomized service for the device combination selected by the user, list detailed and rich use scenarios for the user, guide the user to fully explore more application scenarios of the function of the super terminal, and achieve the effect of recommending the scene service according to the device combination.

[0513] In combination with the foregoing embodiments, the function modules of the communication system 10 provided by the embodiments of the present application are described below.

[0514] For example, the first device is a smart television, the second device is a smart sound box, and the fourth device is a smart air conditioner. The first scene service option is a scene service option of the smart television, the smart sound box and the smart air conditioner being connected to form a super terminal, and the third scene service option is a scene service option of the smart television, the smart sound box and the smart air conditioner being connected to form a super terminal. Figure 9 , Figure 21The functional modules of the communication system 10 are shown. The communication system 10 may include a device combination generation module 2101, a scene service decision module 2102, a scene service provision module 2103, an atomic service provision module 2104, etc. The communication system 10 may also be referred to as a super terminal, which may include a core device and one or more associated devices. For explanations of super terminals, core devices, associated devices, etc., please refer to the foregoing embodiments; further details will not be repeated here.

[0515] The device combination generation module 2101 can be used for combining core devices and associated devices in a super terminal, including enabling the core device to discover surrounding associated devices, indicating the positional or signal strength relationship between the associated devices and the core device on the interface, and connecting the core device and associated devices. The core device can display a super terminal interface, which can show the discovered surrounding associated devices. In some embodiments, the user can drag the associated device icon close to the core device icon; when it reaches a designated area, the associated device icon can snap to the edge of the core device icon, indicating that the core device has established a connection with the associated device. For explanations regarding the connection status of the core device and associated devices, user operations, and schematic interfaces, please refer to [link to relevant documentation]. Figure 21 , Figure 5 , Figure 6 , Figure 15 , Figure 16 The embodiments described above will not be repeated here.

[0516] The scenario service decision module 2102 can analyze the current user's usage scenario based on information such as the type and / or characteristics of the currently connected devices, product positioning, and commonly used functions, or based on the detected environment of the user or the operating status of the devices. The scenario service decision module 2102 can classify various consumer usage scenarios. In one example, consumer usage scenarios can include the following five categories: mobile office, smart home, audio-visual entertainment, sports and health, and smart travel. Of course, the classification of consumer usage scenarios is not static; the scenario service decision module 2102 can be continuously updated, expanded, or refined to include more usage scenarios. This embodiment does not impose limitations, and specific changes can be made according to actual circumstances. For example, adding previously unseen associated devices or functions may lead to new usage scenarios. For detailed explanations, please refer to the embodiments described above; further details will not be repeated here.

[0517] The scenario service module 2103 is used to comprehensively consider one or more scenario-based service options when multiple devices are connected, based on factors such as the device type, device characteristics, product positioning, device purpose, collaborative services that the device can provide, the current environment of the device, the device operating status, the user's recently used applications, the currently running applications, and the location information entered by the user operation. These options are then displayed on the user interface to facilitate users in quickly launching relevant services or functions.

[0518] In some embodiments, when listing scenario-based service options, the scenario service providing module 2103 can calculate the probability of each scenario-based service option being selected based on the user's recent frequency of use of different collaborative services, the current device environment, or the user's recently run applications, and prioritize displaying scenario-based service options with a high probability of selection in the scenario-based service option area. Of course, scenario-based service options are not static; the scenario service providing module 2103 can continuously update, expand, or refine more scenario-based service options. This embodiment does not impose limitations, and specific changes can be made according to actual circumstances. In some embodiments, users can customize and add scenario-based services, such as setting a name for a usage scenario and associating that scenario with other application services or system services. The newly added scenario-based service option is displayed alongside other scenario-based service options in the super terminal interface. The scenario service providing module 2103 can comprehensively consider one or more of the above factors to reasonably recommend scenario-based service options.

[0519] The scenario service provider module 2103 can also be used to recommend associated devices. Based on the types of devices currently constituting the super terminal system, the scenario service provider module 2103 can analyze the current scenario and list and recommend other devices suitable for the current scenario to help users obtain a better scenario experience; or it can infer more expandable use scenarios and then list and recommend other devices to the user. If the user adopts the devices recommended by the system, more and richer use scenarios can be further expanded.

[0520] For a description of the functions that module 2103, which provides more scenario services, can offer, please refer to [link / reference]. Figure 17 , Figure 7 , Figure 8 , Figure 9 , Figure 11 , Figure 12 , Figure 13A , Figure 13B , Figure 13C , Figure 13D , Figure 15 , Figure 16 , Figure 17 , Figure 18 The descriptions of the aforementioned embodiments will not be repeated here.

[0521] The atomic service providing module 2104 can be used to automatically set the atomic service combination in the scene based on the current device combination, the atomic services provided by each device queried, the actual selected service in the scene by the user, or the commonly used service combination of the user, or the default service combination set by the user, or the most suitable service combination in the scene based on life cognition, or the service combination with the most frequent use in the scene obtained from the server, etc., so as to save the user from the tedious selection operation. The atomic service refers to the smallest unit of capability that can be independently run, and is a concept of abstract encapsulation of a single function / capability, which can be a hardware service or a software service.

[0522] In some embodiments, the atomic service providing module 2104 supports the user to set the atomic service combination individually, and the user can manually modify the atomic service combination according to personal preferences or actual conditions, and adjust a certain atomic service to be provided by another device. For example, in a cross-screen conference, if the surrounding environment is relatively noisy, the user can change the audio output function from being provided by the smart TV to being provided by the earphone.

[0523] For more functional description of the atomic service providing module 2104, please refer to the description of the foregoing embodiments, which will not be repeated here. Figure 19

[0524] In some embodiments, the electronic devices that can join the super terminal can include same-account devices, cross-account devices, and no-account devices.

[0525] The same-account device can include an electronic device whose login account is the same as the login account of the core device of the super terminal. For example, the login account of a mobile phone of Zhang San and a tablet computer of Zhang San is the same account, i.e., the account of Zhang San. In the case that the core device of the super terminal is the mobile phone of Zhang San, the tablet computer of Zhang San can join the super terminal as a same-account device.

[0526] ​The same-account device can further include an electronic device paired with the core device of the super terminal and associated with the account logged in on the core device. In some embodiments, some electronic devices (such as earphones, smart watches, smart bands, etc.) do not have the ability to log in an account independently. The user can pair the electronic device with a mobile phone (or a tablet computer or other electronic device with the ability to log in an account independently), thereby associating the electronic device with the account logged in on the mobile phone, and facilitating the control and management of the electronic device through the mobile phone. The pairing method can include Bluetooth pairing, Wi-Fi pairing, etc. The embodiments of the present application do not limit the pairing method. For example, Zhang San's earphones do not have the ability to log in an account independently. Zhang San's mobile phone is paired with Zhang San's earphones. Then, Zhang San's earphones are associated with the account logged in on Zhang San's mobile phone (i.e., Zhang San's account). That is, Zhang San's earphones can be an electronic device managed under Zhang San's account. In the case where the core device of the super terminal is the above-mentioned mobile phone of Zhang San, Zhang San's earphones can join the super terminal as a same-account device.

[0527] The cross-account device can include an electronic device whose logged-in account is different from the logged-in account of the core device of the super terminal. For example, the account logged in on Zhang San's mobile phone is Zhang San's account. The account logged in on Li Si's tablet computer is Li Si's account. The account logged in on Zhang San's mobile phone is different from the account logged in on Li Si's tablet computer. In the case where the core device of the super terminal is the above-mentioned mobile phone of Zhang San, Li Si's tablet computer can join the super terminal as a cross-account device.

[0528] The cross-account device can further include an electronic device associated with an account by pairing with other electronic devices, and the associated account is different from the logged-in account of the core device of the super terminal. For example, the account logged in on Li Si's mobile phone is Li Si's account. Li Si's earphones are paired with Li Si's mobile phone, thereby being associated with Li Si's account. In the case where the core device of the super terminal is the above-mentioned mobile phone of Zhang San (the logged-in account is Zhang San's account), Li Si's earphones can join the super terminal as a cross-account device.

[0529] The accountless device can be an electronic device without a logged-in account and without an associated account. After other electronic devices (such as mobile phones, tablet computers, etc.) establish a communication connection with the accountless device, the accountless device can work cooperatively with the other electronic devices to provide scenario-based services for the user. The accountless device can be equivalent to a public device. For example, a printer does not have a logged-in account and does not have an associated account. In the case where the core device of the super terminal is the above-mentioned mobile phone of Zhang San (the logged-in account is Zhang San's account), the printer can join the super terminal as an accountless device.

[0530] The embodiments of the present application do not limit the types of the same-account device, the cross-account device, and the accountless device.

[0531] In some embodiments, the cross-account device can be bound to the core device of the super terminal before joining the super terminal. The core device of the super terminal can request to bind with the cross-account device. After the cross-account device agrees to bind, the core device can bind with the cross-account device, so that the cross-account device can join the super terminal. The super terminal can provide scenario-based services for the user by using the cross-account device.

[0532] It can be understood that the user to which the cross-account device belongs is usually not the same as the user to which the core device of the super terminal belongs. The user to which the core device belongs wants to provide scenario-based services by using the cross-account device, and needs to obtain the consent of the user to which the cross-account device belongs. Therefore, when the cross-account device joins the super terminal, the core device can bind with the cross-account device to request the consent of the user to which the cross-account device belongs to use the cross-account device by the user to which the core device belongs. In this way, the use of the electronic device of the user to which the cross-account device belongs by the user to which the core device belongs in the case of disagreement or unawareness can be reduced, and the user experience can be improved.

[0533] In some embodiments, the same-account device can join the super terminal without the device binding process. It can be understood that the user to which the same-account device belongs is usually the same as the user to which the core device of the super terminal belongs. The user to which the core device belongs wants to use other devices to cooperate with the core device, and can make the core device cooperate with the same-account device to provide scenario-based services by the method shown in the foregoing Figure 10 In this way, the user can use multiple electronic devices to cooperate without device binding. This can simplify the operation of the user to use the super terminal to provide scenario-based services, and improve the user experience.

[0534] In some embodiments, the no-account device can join the super terminal with or without device binding.

[0535] The following specifically introduces a scenario in which an electronic device joins a super terminal and is bound to a device.

[0536] Figure 5 to Figure 8 An example of a scenario in which an electronic device joins a super terminal and is bound to a device is shown.

[0537] As shown in Figure 22A to Figure 22H , the electronic device 100 can display a user interface 2210. Here, the electronic device 100 is taken as a mobile phone as an example for description. The electronic device 100 can also be other types of electronic devices (such as a tablet computer, a notebook computer, etc.) without being limited to a mobile phone. The user interface 2210 can refer to the user interface 2100 shown in the foregoing Figure 22AIntroduction of the illustrated user interface 500.

[0538] The user interface 2210 can include a device connection status diagram 2211 and an add control 2212. The device connection status diagram 2211 can include a phone icon 505. The phone icon 505 can represent that the core device is a phone, i.e., the electronic device 100. The device icons and names around the phone icon 505 represent the electronic devices found by the electronic device 100. For example, the Sound X icon, the FreeBuds of Zhang San icon, the MatePad of Zhang San icon, and the MateBook of Zhang San icon in the device connection status diagram 2211.

[0539] In some embodiments, when an account is logged in on the electronic device, the name of the electronic device can be the account name + the type and / or model of the electronic device. When other electronic devices find the electronic device, the name of the electronic device can be displayed. For example, the type of the electronic device is MatePad. The account is logged in on the electronic device, and the account name is “Zhang San”. The name of the electronic device can be the MatePad of Zhang San. The electronic device 100 finds the electronic device, and the device connection status diagram 2211 can display the MatePad of Zhang San icon and the name of the electronic device “the MatePad of Zhang San”. Figure 5 The device connection status diagram 2211 illustrated can display the MatePad of Zhang San icon and the name of the electronic device “the MatePad of Zhang San”.

[0540] In some embodiments, the name of the electronic device can be user remarks. The user can manage the electronic devices paired with the electronic device 100 through the electronic device 100, for example, modify the name of the electronic devices paired with the electronic device 100. For example, the electronic device 100 is paired with the earphone. The user can modify the name of the earphone to the FreeBuds of Zhang San through the electronic device 100. The electronic device finds the earphone, and the device connection status diagram 2211 can display the FreeBuds of Zhang San icon and the name of the electronic device “the FreeBuds of Zhang San”. Figure 22A The device connection status diagram 2211 illustrated can display the FreeBuds of Zhang San icon and the name of the electronic device “the FreeBuds of Zhang San”.

[0541] The embodiments of the present application do not limit the method of determining the name of the electronic device described above...

Claims

1. A device interaction method, applied to a first device, characterized in that, The method includes: The first interface is displayed, which includes a first device control, a second device control, and a third device control. The first device control is used to indicate the first device, the second device control is used to indicate the second device, and the third device control is used to indicate the third device. When the first device displays the first interface, it receives a first user operation, which is used to associate the first device and the second device. The first device displays a second interface, which includes a first service option for a first scenario-based service and a second service option for a second scenario-based service. The first service option is used to perform a screen mirroring function. The second scenario-based service is different from the first scenario-based service. The interface displayed when the first device performs the screen mirroring function is different from the interface displayed when performing the second scenario-based service. Receive a second user action on the first service option; In response to the second user's operation, the screen of the first device is projected onto the second device; When the first device projects its screen onto the second device, the first device displays the first control; The first device receives a third user operation on the first control, and displays a remote control interface for controlling the second device. When the first device displays the first interface, it receives a fourth user operation, which is used to associate the first device and the third device. The first device displays a third interface, which includes a third service option for a third scenario-based service. The third scenario-based service is different from the first scenario-based service and different from the second scenario-based service.

2. The method according to claim 1, characterized in that, The remote control interface includes any one of the following: homepage control, switch control, volume up control, volume down control, back control, menu control, voice control, confirmation control, and up / down / left / right directional controls.

3. The method according to claim 1, characterized in that, The first user operation is an operation on the second device control.

4. The method according to claim 3, characterized in that, The first device displays a second interface including: In response to the first user action, the second interface is displayed.

5. The method according to claim 1 or 2, characterized in that, The first device includes any one of the following: a mobile phone or a tablet computer; The second device is a smart TV.

6. The method according to claim 1, characterized in that, The first user operation is a drag-and-drop operation, and the method further includes... The position where the screen of the first device is projected onto the second device is determined based on the relative positions of the first device control and the second device control during the release and drag operation.

7. The method according to claim 6, characterized in that, The step of determining the position of the screen projection from the first device to the second device based on the relative positions of the first device control and the second device control during the release and drag operation specifically includes: If the relative position is such that the second device control is to the left of the first device control, then the screen of the first device is projected onto the right side of the second device. If the relative position is such that the second device control is to the right of the first device control, then the screen of the first device is projected onto the left side of the second device. If the relative position is such that the second device control is in the middle of the first device control, then the screen of the first device is cast to the second device in full screen.

8. An electronic device, characterized in that, The electronic device includes: a communication device, a display device, a memory, and a processor coupled to the memory, as well as one or more programs; the communication device is used for communication, the display device is used for displaying an interface, the memory stores computer-executable instructions, and the processor executes the instructions to cause the electronic device to implement the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1 to 7.

10. A communication system, the communication system comprising a first device, a second device, and a third device, wherein, The first device performs the method as described in any one of claims 1 to 7.

Images