Cross-end interaction method and related equipment thereof

Abstract

The invention provides a cross-end interaction method and related equipment thereof, and relates to the technical field of terminals. According to the method, when a first electronic device displays a first material, a target application is started and a hidden state icon is displayed through a first stage of a first operation; collecting the first material into the target application through a second stage of the first operation, and switching the hidden state icon into a collection state icon; after the first material is collected, the collection state icon is switched to the hidden state icon; before the first material is collected, the second electronic equipment displays a hidden icon; after the first electronic device is collected, the hidden-state icon on the second electronic device is switched into a suspended-state icon through a third operation, and at the moment, the suspended-state icon comprises a first icon indicating the first material and a first corner mark indicating that the first material comes from other electronic devices except the second electronic device. According to the method and the device, the materials are collected on one device, and the icons of the materials are automatically synchronized on other devices, so that the cross-end interaction efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a cross-terminal interaction method and related equipment. Background Art

[0002] With the continuous development of terminal technology, electronic devices can support the collection of different types of materials in different forms. For example, taking a mobile phone as an example, users can use the mobile phone to copy and save text, capture pictures, record audio and video, etc., and can also use it to collect web pages and videos obtained from some applications (for example, news applications, etc.). In addition, electronic devices can also support the organization of materials. For example, users can copy, edit and delete the collected materials through their mobile phones.

[0003] In actual processing, the process of collecting and organizing materials not only involves a variety of collection forms and material types, but also involves multiple applications, multiple collections and multiple arrangements. Moreover, with the continuous enrichment of terminal types and the widespread application of multi-device collaboration, it is more likely to involve cross-terminal processing. In this regard, the corresponding methods currently provided, such as file management applications and note-taking applications, are scattered, limited, and not convenient enough, which seriously affects the user experience.

[0004] Therefore, how to improve the efficiency of cross-terminal material processing through electronic devices is an urgent problem to be solved. Summary of the invention

[0005] The present application provides a cross-terminal interaction method and related devices. According to the method, electronic devices can collect and synchronize different types of materials across terminals to a global collection application, and can also perform secondary sorting and centralized one-time use on the collected materials. This method can greatly improve the efficiency of collecting, sorting and sharing materials, thereby improving the user experience.

[0006] In a first aspect, the present application provides a cross-end interaction method, which can be applied to a cross-end interaction system including a first electronic device and a second electronic device. The method may include:

[0007] The first electronic device displays a first interface, where the first interface includes a first material;

[0008] In the first electronic device, in response to the first stage of the first operation on the first material, a target application is started, and a hidden icon corresponding to the target application is suspended and displayed on the first interface;

[0009] In the first electronic device, in response to the second stage of the first operation on the first material, the hidden state icon is switched to a collection state icon corresponding to the target application, and the first material is collected into the target application through the collection state icon; the size of the hidden state icon is smaller than the size of the collection state icon, the collection state icon is displayed in a floating manner on the first interface, and the first stage and the second stage are continuous;

[0010] In the first electronic device, after the first material is collected, the collection state icon is switched to the hidden state icon;

[0011] In the first electronic device, in response to a second operation on the collected hidden state icon, the hidden state icon is switched to a floating state icon corresponding to the target application, the floating state icon is displayed in a floating state on the first interface, and the floating state icon includes a first icon indicating the first material;

[0012] Before the first material is collected, the second electronic device displays a second interface, wherein the hidden icon corresponding to the target application is suspended and displayed on the second interface;

[0013] After the first material is collected, the second electronic device detects a third operation on the hidden icon in the second interface;

[0014] In the second electronic device, in response to the third operation, the hidden state icon is switched to the floating state icon, and the floating state icon is displayed floating on the second interface. The floating state icon includes the first icon and a first superscript indicating the first material, and the first superscript is used to indicate that the first material comes from an electronic device other than the second electronic device.

[0015] In some embodiments of the present application, the first electronic device is a mobile phone, and the second electronic device is a tablet computer or a desktop computer.

[0016] The target application is a global favorite application in the embodiment of the present application. The second electronic device displays a hidden icon indicating that the target application has been started.

[0017] In some embodiments of the present application, the collection process of the first material may indicate Figure 3A-Figure 3F The text collection process shown.

[0018] In some embodiments of the present application, the process of viewing the first icon at the first electronic device may indicate Figure 3G-3H The process shown.

[0019] In some embodiments of the present application, the process of viewing the first icon on the second electronic device may indicate Figure 3I-3JThe process shown.

[0020] In the solution provided in the present application, when the first electronic device displays the first material, it starts the target application and displays the hidden state icon through the first stage of the first operation; through the second stage of the first operation, the first material is collected into the target application, and the hidden state icon is switched to the collection state icon; after the first material is collected, the collection state icon is switched to the hidden state icon; the second operation is performed on the hidden state icon, and the first icon corresponding to the first material can be seen in the switched floating state icon.

[0021] At the same time, before the first material is collected, the second electronic device displays a hidden icon; and after the first material is collected, through a third operation, the hidden icon on the second electronic device will switch to a floating icon. At this time, the floating icon includes a first icon indicating the first material and a first superscript indicating that the first material comes from an electronic device other than the second electronic device. The present application can realize the collection of materials on the first electronic device, and automatically synchronize the icon of the material in other devices in the ring (such as the second electronic device), so that synchronization can be achieved without manual interaction, thereby improving the efficiency of cross-end interaction. In addition, when the material icon is displayed on the collecting device, there is no superscript, and when the material icon is displayed on the synchronized device, it carries a superscript. Through this distinction, the user can intuitively see the source of the material, thereby improving the user experience.

[0022] In combination with the first aspect, in a possible implementation manner, the floating icon in the second electronic device includes a first control, and after the floating icon is displayed in a floating state on the second interface, the method further includes:

[0023] In the second electronic device, in response to a fourth operation on the first control, the second interface suspends and displays a first card, where the first card at least includes a plurality of callable service options corresponding to the first electronic device, and the callable service options include a shooting option;

[0024] In the second electronic device, in response to a fifth operation on the shooting option, the second electronic device sends a first instruction to the first electronic device, where the first instruction is used to instruct to start a camera application of the first electronic device;

[0025] After receiving the first instruction, the first electronic device starts the camera application and displays a shooting interface;

[0026] In the second electronic device, the first card is switched to a multi-screen collaborative interface, and the multi-screen collaborative interface is displayed floating on the second interface, and the multi-screen collaborative interface includes the shooting interface corresponding to the first electronic device.

[0027] In some embodiments of the present application, the first control indicates a "more" control.

[0028] In some embodiments of the present application, the shooting options may include a photo taking option and a video recording option.

[0029] In combination with the first aspect, in a possible implementation manner, the shooting interface includes a shooting control, and the method further includes:

[0030] In the second electronic device, in response to a sixth operation on the shooting control included in the multi-screen collaborative interface, a second instruction is sent to the first electronic device, where the second instruction is used to instruct shooting;

[0031] After receiving the second instruction, the first electronic device shoots and collects the shot second material into the target application of the first electronic device;

[0032] After the second material is collected, the floating icon of the first electronic device includes a second icon indicating the second material;

[0033] The first electronic device sends the second icon and a second superscript to the second electronic device, where the second superscript is used to indicate that the second material comes from another electronic device except the second electronic device.

[0034] In some embodiments of the present application, the shooting control may include a photo control and a video recording control.

[0035] In some embodiments of the present application, the second electronic device triggers a cross-terminal shooting option; calls the shooting function on the first electronic device to capture the second material; the second material is an image or a video. Figure 4A-4B , Figure 4C-4F The diagram shows the process of calling the camera service on the mobile phone from the desktop computer to take pictures.

[0036] According to this implementation method, the second electronic device can call the camera service on the first electronic device across terminals to take pictures and collect images or videos. Then, the first electronic device displays the second icon and sends it to the second electronic device.

[0037] It is understandable that when the screen of the first electronic device is locked, the second electronic device can also call the camera service to take pictures without unlocking the screen.

[0038] It is understandable that the first electronic device may only send the second icon to the second electronic device, and the second electronic device may add a second superscript to mark the second icon when receiving the second icon.

[0039] In combination with the first aspect, in a possible implementation manner, the callable service option further includes a recording option, and the method further includes:

[0040] In the second electronic device, in response to the seventh operation on the recording option, the second electronic device sends a third instruction to the first electronic device, where the third instruction is used to instruct to start a recording application of the first electronic device;

[0041] After receiving the third instruction, the first electronic device starts the recording application and displays a recording interface;

[0042] In the second electronic device, the first card is switched to the multi-screen collaborative interface, and the multi-screen collaborative interface is displayed floating on the second interface. The multi-screen collaborative interface includes the recording interface corresponding to the first electronic device.

[0043] In combination with the first aspect, in a possible implementation manner, after receiving the third instruction, the first electronic device starts the recording application and displays a recording interface, including:

[0044] The first electronic device displays a lock screen interface;

[0045] In the first electronic device, in response to an unlocking operation on the lock screen interface, starting the recording application and displaying the recording interface;

[0046] In the second electronic device, the multi-screen collaborative interface includes the recording interface corresponding to the first electronic device, including:

[0047] When the first electronic device displays the lock screen interface, the multi-screen collaboration interface in the second electronic device displays a lock screen prompt interface, where the lock screen prompt interface is used to prompt that the first electronic device is locked;

[0048] When the first electronic device starts the recording application and displays the recording interface, in the second electronic device, the lock screen prompt interface included in the multi-screen collaborative interface is switched to the recording interface corresponding to the first electronic device.

[0049] In combination with the first aspect, in a possible implementation manner, the recording interface includes a recording control, and the method further includes:

[0050] In the second electronic device, in response to an eighth operation on the recording control included in the multi-screen collaborative interface, sending a fourth instruction to the first electronic device, where the fourth instruction is used to instruct recording;

[0051] After receiving the fourth instruction, the first electronic device records the audio and collects the third material obtained by the recording into the target application of the first electronic device;

[0052] After the third material is collected, the floating icon of the first electronic device includes a third icon indicating the third material;

[0053] The first electronic device sends the third icon and the third superscript to the second electronic device, where the third superscript is used to indicate that the third material comes from another electronic device except the second electronic device.

[0054] In some embodiments of the present application, the second electronic device triggers a cross-terminal recording option; the recording function on the first electronic device is called to collect a third material; the third material is audio. Figure 5A-5H The diagram shows the process of calling the audio service on the mobile phone from the desktop computer to record audio.

[0055] According to this implementation method, the second electronic device can call the audio service on the first electronic device across terminals to record audio and collect audio. Then, the first electronic device displays the third icon and sends it to the second electronic device.

[0056] It is understandable that when the screen of the first electronic device is locked, it is necessary to unlock the first electronic device; only after unlocking can the second electronic device call the audio service to record.

[0057] It is understandable that the first electronic device may only send the third icon to the second electronic device, and when the second electronic device receives the third icon, it adds a third superscript for marking.

[0058] In combination with the first aspect, in a possible implementation manner, the callable service information further includes a handwriting option, and the method further includes:

[0059] In the second electronic device, in response to a ninth operation on the handwriting option, the second electronic device sends a fifth instruction to the first electronic device, where the fifth instruction is used to instruct to start a note application of the first electronic device;

[0060] After receiving the fifth instruction, the first electronic device starts the note application and displays a handwriting input interface;

[0061] In the second electronic device, the first card is switched to the multi-screen collaborative interface, and the multi-screen collaborative interface is displayed floating on the second interface. The multi-screen collaborative interface includes the handwriting input interface corresponding to the first electronic device.

[0062] In combination with the first aspect, in a possible implementation manner, after receiving the fifth instruction, the first electronic device starts the note application and displays a handwriting input interface, including:

[0063] The first electronic device displays a lock screen interface;

[0064] In the first electronic device, in response to an unlocking operation on the lock screen interface, the first electronic device starts the note application and displays the handwriting input interface;

[0065] In the second electronic device, the multi-screen collaborative interface includes the handwriting input interface corresponding to the first electronic device, including:

[0066] When the first electronic device displays the lock screen interface, the multi-screen collaboration interface in the second electronic device displays a lock screen prompt interface, where the lock screen prompt interface is used to prompt that the first electronic device is locked;

[0067] When the first electronic device starts the note application and displays the handwriting input interface, in the second electronic device, the lock screen prompt interface included in the multi-screen collaboration interface is switched to the handwriting input interface corresponding to the first electronic device.

[0068] In combination with the first aspect, in a possible implementation manner, the handwriting input interface includes a handwriting input area and a completion control, and the method further includes:

[0069] In the second electronic device, in response to an input operation on the handwriting input area included in the multi-screen collaborative interface, sending handwriting input content to the first electronic device;

[0070] In response to a tenth operation on the completion control, sending a sixth instruction to the first electronic device, wherein the sixth instruction is used to indicate that the handwriting input is completed;

[0071] After receiving the sixth instruction, the first electronic device collects the fourth material obtained according to the received handwriting input content into the target application of the first electronic device;

[0072] After the fourth material is collected, the floating icon of the first electronic device includes a fourth icon indicating the fourth material;

[0073] The first electronic device sends the fourth icon and the fourth superscript to the second electronic device, where the fourth superscript is used to indicate that the fourth material comes from another electronic device except the second electronic device.

[0074] In some embodiments of the present application, the second electronic device triggers a handwriting option across terminals; the handwriting function on the first electronic device is called to collect a fourth material; the fourth material is a note. Figure 6A-6D The diagram shows the process of calling the handwriting input service on the mobile phone from the desktop computer to perform handwriting.

[0075] According to this implementation method, the second electronic device can call the handwriting input service on the first electronic device across terminals to perform handwriting and collect notes. Then, the first electronic device displays the fourth icon and sends it to the second electronic device.

[0076] It is understandable that when the screen of the first electronic device is locked, it is necessary to unlock the first electronic device; only after unlocking can the handwriting input service be called by the second electronic device to record.

[0077] It is understandable that the first electronic device may only send the fourth icon to the second electronic device, and the second electronic device may add a fourth superscript to mark the fourth icon when receiving the fourth icon.

[0078] In combination with the first aspect, in a possible implementation manner, the multi-screen collaborative interface further includes an exit control, and the method further includes:

[0079] In the second electronic device, in response to an eleventh operation on the exit control, the multi-screen collaborative interface is closed and switched to the floating state icon, and the floating state icon is displayed in a floating state on the second interface;

[0080] The floating state icons of the second electronic device include a second icon and a second superscript, or include a third icon and a third superscript; or include a fourth icon and a fourth superscript.

[0081] After the second electronic device exits the multi-screen collaborative interface, a second icon and a second superscript, a third icon and a third superscript, and a fourth icon and a fourth superscript may be displayed on the second electronic device to distinguish the device source of the material through the superscript.

[0082] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0083] Before the first material is collected, the second electronic device displays a second interface, the second interface includes a sidebar displayed in a suspended manner, and the sidebar includes an application icon corresponding to the target application;

[0084] After the first material is collected, the second electronic device detects a twelfth operation on the application icon;

[0085] In the second electronic device, in response to the twelfth operation, the target application is started, the sidebar is switched to the floating icon, the floating icon is displayed in a floating state on the second interface, and the floating icon includes the first icon and the first superscript;

[0086] When the second electronic device starts the target application for the first time, the floating icon further includes a prompt card, and the prompt card is used to prompt that the first material comes from another electronic device except the second electronic device;

[0087] In the second electronic device, in response to a thirteenth operation on the top horizontal bar control included in the floating state icon, the floating state icon is switched to the hidden state icon, and the hidden state icon is displayed in a floating state on the second interface.

[0088] It is understandable that before the first electronic device collects the first material, the target application on the second electronic device may not be started. At this time, the second electronic device may display a sidebar including an application icon indicating the target application.

[0089] In some embodiments of the present application, the first electronic device collects the first material, the second electronic device starts the target application for the first time, and the process of viewing the first icon from the first electronic device can indicate Figure 7G-Figure 7I The process shown.

[0090] According to this implementation, when the target application on the second electronic device is opened for the first time, the user can be prompted to learn the icon synchronization method through a prompt card.

[0091] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0092] The second electronic device displays a third interface, the third interface includes the floating icon displayed in a floating state, the floating icon includes the first icon, the first superscript, the second icon and the second superscript;

[0093] In the second electronic device, in response to the fourteenth operation, after the first icon moves to the second icon position or after the second icon moves to the first icon position, the first icon and the second icon switch to a first group of icons;

[0094] In the first electronic device, the first icon and the second icon are synchronously switched to the first group of icons.

[0095] In some embodiments of the present application, the fourteenth operation may be a drag operation, and the process of switching the first icon and the second icon to the first group of icons may indicate Figure 8A-8F The process of dragging to group is shown.

[0096] According to this implementation, multiple materials can be dragged into groups, and the grouping results can be synchronized to other devices.

[0097] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0098] The second electronic device displays a fourth interface, the fourth interface includes the floating icon displayed in a floating state, the floating icon includes the first icon, the first superscript, the second icon and the second superscript;

[0099] In the second electronic device, in response to a fifteenth operation, the first icon or the second icon moves within the floating icon;

[0100] In the first electronic device, the first icon or the second icon moves synchronously.

[0101] In some embodiments of the present application, the fifteenth operation may be a drag operation, and the movement process of the first icon or the second icon may indicate Figure 9A-9E The moving process is shown.

[0102] According to this implementation, the position of each icon can be moved, and the movement result can be synchronized to other devices.

[0103] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0104] The second electronic device displays a full-screen interface corresponding to the target application; the first icon included in the full-screen interface is in a selected state, and the full-screen interface also includes a rename control;

[0105] In the second electronic device, in response to a sixteenth operation on the rename control, a fifth interface is displayed, wherein the fifth interface includes an input field;

[0106] In the second electronic device, in response to an input operation on the input field, naming the first icon as a target name;

[0107] In the second electronic device, after naming, the full-screen interface includes the first icon whose name is the target name;

[0108] In the first electronic device, the name of the first icon is synchronized with the target name.

[0109] In some embodiments of the present application, the sixteenth operation may be a click operation, and the process of renaming the first icon may indicate Figures 10A-10H The renaming process is shown.

[0110] According to this implementation, each icon can be renamed, and the naming result can be synchronized to other devices.

[0111] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0112] The second electronic device displays a full-screen interface; the first icon and the second icon in the full-screen interface are in a selected state, and the full-screen interface further includes a delete control;

[0113] In the second electronic device, in response to a seventeenth operation on the deletion control, deleting the first material and the second material;

[0114] In the second electronic device, after the deletion, the full-screen interface does not include the first icon and the second icon;

[0115] In the first electronic device, the first material and the second material are deleted synchronously, and after the deletion, the first icon and the second icon are not included.

[0116] In some embodiments of the present application, the seventeenth operation may be a click operation, and the process of deleting the first icon and the second icon may indicate Figures 11A-11F Deletion process shown.

[0117] According to this implementation, multiple icons can be deleted, and the deletion results can be synchronized to other devices.

[0118] It can be understood that when deleting, the icon and the material corresponding to the icon are deleted together.

[0119] In combination with the first aspect, in a possible implementation manner, after the floating icon includes a first icon indicating the first material and the first superscript, the method further includes:

[0120] In the second electronic device, in response to the interface switching operation on the second electronic device, the second electronic device displays a sixth interface, and the floating icon is displayed in a floating state on the sixth interface;

[0121] In the second electronic device, in response to the eighteenth operation on the first icon, the second electronic device downloads the first material indicated by the first icon and displays a seventh interface, the seventh interface includes the first material, and the floating icon is displayed floating on the seventh interface, and the floating icon does not include the first icon and the first superscript.

[0122] In some embodiments of the present application, the eighteenth operation may be a drag operation, and the process of downloading and using the first material may indicate Figures 12A-12F The process shown.

[0123] According to this implementation method, by dragging the first icon to a certain application interface, the first material corresponding to the first icon can be downloaded and used, and after use, the second electronic device will not include the first icon and the first superscript, and the result can be synchronized to other devices.

[0124] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0125] The second electronic device displays a full-screen interface; the first icon and the fifth icon in the full-screen interface are in an icon selected state, and the full-screen interface also includes a sharing control; the fifth icon is used to indicate the fifth material collected by the target application in the second electronic device;

[0126] In the second electronic device, in response to a nineteenth operation on the sharing control, a second card is displayed, wherein the second card includes a plurality of options indicating different formats, different devices, different applications, saving, and printing;

[0127] In the second electronic device, in response to a twentieth operation on the second card, the first material and the fifth material are shared based on the selected option.

[0128] In some embodiments of the present application, the above-mentioned sharing process may indicate Figures 13A-13E The material sharing process shown.

[0129] According to this implementation, different sharing can be performed based on the options on the second card.

[0130] In combination with the first aspect, in a possible implementation manner, after responding to a nineteenth operation on the sharing control and before displaying a third card, the method further includes:

[0131] displaying a third card, the third card comprising a merge option;

[0132] In response to a twenty-first operation on the merge option, merge the first material and the fifth material;

[0133] The first material and the fifth material are shared based on the selected option, including:

[0134] The merged first material and the fifth material are shared based on the option selected.

[0135] In some embodiments of the present application, the above-mentioned sharing process may indicate Figures 13A-13E The material sharing process shown.

[0136] According to this implementation method, the materials can be merged before sharing.

[0137] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0138] In the second electronic device, in response to a twenty-second operation on the first icon, the first material is downloaded and displayed.

[0139] In some embodiments of the present application, a full-screen preview interface corresponding to the first material may be downloaded and displayed, or the first material may be downloaded and displayed in a floating window. The process of triggering the display of other materials is similar and will not be described in detail here.

[0140] In combination with the first aspect, in a possible implementation manner, in the first electronic device, switching the collection state icon to the hidden state icon includes:

[0141] The collection state icon is switched to a feedback state icon corresponding to the target application, and the feedback state icon is used to indicate that the collection has been successful;

[0142] The feedback state icon is switched to the hidden state icon.

[0143] According to this implementation method, the collected dynamics can be fed back in real time to improve the user experience.

[0144] In combination with the first aspect, in a possible implementation, the type of the first material includes at least: text, graphics, images, audio, and tables; the first stage of the first operation is a long press operation, and the second stage of the first operation is dragging or throwing.

[0145] According to this implementation method, different types of materials can be collected based on the same operation method, reducing the learning difficulty for users.

[0146] In combination with the first aspect, in a possible implementation manner, the hidden state icon is in a semi-transparent semicircular shape, and the collection state icon is in a capsule shape or a window shape.

[0147] According to this implementation, interference with the user's vision and hindrance to the user's operation can be avoided.

[0148] In combination with the first aspect, in a possible implementation manner, the first superscript, the second superscript, the third superscript and the fourth superscript are the same.

[0149] According to this implementation, the display effect can be simplified, and only whether it comes from the local device is distinguished.

[0150] In a second aspect, the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute a method involved in the first electronic device or the second electronic device in the first aspect or any one of the implementations of the first aspect.

[0151] In a third aspect, the present application provides a cross-end interaction system, which includes a first electronic device and a second electronic device, and the first electronic device and the second electronic device execute the method involved in the first aspect or any one implementation of the first aspect.

[0152] In a fourth aspect, the present application provides a computer storage medium, which stores a computer program. When the computer program is executed by an electronic device, the electronic device executes the method involved in the first aspect or any one of the implementations of the first aspect.

[0153] In the fifth aspect, an embodiment of the present application provides a chip system, which can be applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0154] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0155] It can be understood that the electronic device in the fourth aspect, the fifth aspect and the sixth aspect can indicate the electronic device provided by the second aspect, that is, can indicate the first electronic device or the second electronic device in the first aspect.

[0156] It is understandable that the electronic device provided in the second aspect, the cross-end interactive system provided in the third aspect, the computer storage medium provided in the fourth aspect, the chip system provided in the fifth aspect, and the computer program product provided in the sixth aspect are all used to execute the method described in the first aspect or any one of the implementations of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of any possible implementation in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0157] Figure 1 This is a schematic diagram of multiple devices in the same trust ring;

[0158] Figure 2A-2I Schematic diagram of various states of global collection application;

[0159] Figure 3A-3H A set of exemplary user interfaces for collecting and viewing text on this end;

[0160] Figure 3I-3J A set of exemplary user interfaces involved in viewing synchronized text for a trusted device;

[0161] Figure 4A-4B A set of exemplary user interfaces involved in the entry of cross-end invocation of trusted device-related services;

[0162] Figure 4C-Figure 4F A set of exemplary user interfaces involved in cross-end invoking a trusted device camera service;

[0163] Figure 4G-4H A set of exemplary user interfaces for exiting cross-side invocation of trusted device-related services;

[0164] Fig. 4I A schematic diagram of downloading synchronized materials;

[0165] Figure 5A-5D A set of exemplary user interfaces involved in cross-end invocation of trusted device audio services;

[0166] Figure 5E-Figure 5H Another exemplary set of user interfaces involved in cross-side invocation of a trusted device audio service;

[0167] Figure 6A-6D A set of exemplary user interfaces involved in cross-end invocation of a trusted device handwriting service;

[0168] Figure 7A-Figure 7I A set of exemplary user interfaces involved in collecting successful cross-end notifications for the first time;

[0169] Figure 8A-8F A set of exemplary user interfaces involved in dragging a group in a hovering state;

[0170] Figure 9A-9E Another set of exemplary user interfaces involved in dragging and moving in a hovering state;

[0171] Figures 10A-10H Another set of exemplary user interfaces involved in renaming a material in full screen mode;

[0172] Figures 11A-11F A set of exemplary user interfaces involved in deleting material in full screen mode;

[0173] Figures 12A-12F A set of exemplary user interfaces for using the material;

[0174] Figures 13A-13E A set of exemplary user interfaces for sharing materials in full screen mode;

[0175] Fig.14 An exemplary flow chart of a cross-terminal interaction method provided in an embodiment of the present application;

[0176] Fig.15 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0177] Fig.16 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0178] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0179] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0180] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0181] To facilitate understanding, the nouns involved in the embodiments of the present application are first explained.

[0182] 1. Materials in this application may include various types of information such as text, graphics, images, videos, files, audio, web pages, tables, etc.; among which images may be static images or dynamic images, and files may be in various formats such as PDF format, word format, excel format, etc., which are not limited in this application.

[0183] 2. Annotation (or graffiti) in this application refers to the content drawn by the user with a pen tip or fingertip on the display screen. The information contained in the annotation content may include the starting position and ending position of the annotation. The starting position refers to the position where the user puts down the pen when annotating on the display screen, and the ending position refers to the position where the user lifts the pen after annotating on the display screen.

[0184] 3. Trust ring and trusted devices, in this application, refer to multiple electronic devices that have passed trusted authentication and are connected to the same local area network or directly connected (such as establishing a Wi-Fi direct connection or a Bluetooth connection), and have the ability to communicate with each other, that is, these electronic devices form a trust ring. Electronic devices within the trust ring can be collaboratively managed and share resources. In this application, electronic devices within the trust ring can implement functions such as cross-end call services to collect materials, synchronize materials, organize materials, and synchronize the results.

[0185] Electronic devices can perform trusted authentication, establish a trust relationship, and form a trust circle through any of the following methods: 1) Log in to the same account on two or more electronic devices respectively, and the electronic devices under the same account perform trusted authentication through the account to establish a trust relationship and form a trust circle. 2) Two or more electronic devices perform trusted authentication through a private protocol to establish a trust relationship and form a trust circle; 3) Two or more electronic devices perform trusted authentication by logging into the cloud to establish a trust relationship and form a trust circle. The above methods are only examples and are not limited to this in this application.

[0186] It can be understood that the electronic device in the present application can be an electronic device such as a mobile phone, a tablet computer, or a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device (such as a smart watch, a smart bracelet, etc.), a vehicle-mounted device, a smart home device (such as a smart TV, a large screen, etc.) and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0187] For example, Figure 1 As shown in the figure, for example, mobile phones, tablets, desktop computers, laptops, smart TVs and other devices can log in to the same account and access the same local area network to form a trust circle. Within the trust circle, the current task of the mobile phone can be continued on the coordinated devices, such as continuing to watch videos on the mobile phone on the smart screen, and the files on the mobile phone can be easily browsed and edited on the computer.

[0188] A terminal device in a trust circle can be called a trusted device of other devices. In other words, terminal devices that have a trust relationship through trusted authentication are called trusted devices. For example, a mobile phone and a tablet in the same trust circle can be called trusted devices of each other, that is, the tablet is a trusted device of the mobile phone, and the mobile phone is also a trusted device of the tablet.

[0189] The above is a brief introduction to the terms involved in the embodiments of the present application, which will not be repeated below.

[0190] The electronic device can support the collection of different types of materials in different forms, and can also support the arrangement and sharing of the collected materials. In this way, the user can achieve the purpose of collecting materials, arranging materials and sharing materials by using the electronic device.

[0191] However, in actual processing, taking the file management application in the mobile phone as an example, the file management application can only be used to collect materials such as pictures, videos, audio, files, etc. The types of materials are limited, and the forms of collection are also limited accordingly. The materials in the file management application are usually classified according to name, time, size or material type. The organization method is relatively traditional and simple. Correspondingly, it is not very convenient for users to find the individual materials they need. Taking the note application in the mobile phone as an example, in addition to collecting text, pictures, videos, audio, etc., the note application can also be used to collect web pages obtained from news applications, annotations obtained from reading applications, etc. Among them, the web page can be collected in the note application by sliding down with three fingers; the annotation content can be collected in the note application when receiving the user's save instruction.

[0192] In the above two examples, although the note-taking application can collect slightly more types of materials than the file management application, in the process of organizing notes, if multiple types of materials need to be organized into one note, the user needs to frequently switch between different applications during the collection process. At the same time, it is necessary to collect materials in different applications in different collection methods. For users, the learning cost is high and the collection efficiency is low. The materials collected between different electronic devices are also isolated from each other, which makes transmission troublesome. It is very inconvenient when users need to use different materials on multiple electronic devices across terminals.

[0193] In view of this, an embodiment of the present application provides a cross-terminal interaction method. The cross-terminal interaction method can be used to realize cross-terminal collection of materials from different sources, different types of materials, and multiple times, shorten the user's collection path, unify the collection method, and improve the collection efficiency; and can also quickly classify the collected materials according to the user's personal needs, and perform secondary sorting, sharing, multi-terminal synchronization and calling, etc.

[0194] In addition, the related technologies all provide independent organizing tools, which are not closed-loop in terms of scenarios. Compared with the existing technologies, this application provides a closed-loop tool that can connect various scenarios, which is easy to operate, has low learning costs, and can also meet the user's personal usage needs and habits, thereby improving the user's usage experience.

[0195] It should be understood that the materials involved in this application include but are not limited to text, graphics, images, videos, audio, web pages, tables, etc. The material collection involved in this application may include acquiring materials, processing the acquired materials and using the processed results as the collected content. For example, collecting materials may include: converting the acquired audio materials into text materials, converting the acquired text materials into audio materials, editing or marking the input text or pictures, etc. Organizing materials may include: grouping, moving, renaming, deleting, etc. for different types of materials. Sharing materials may include: sharing content in different formats, printing or sharing to third-party applications, etc.

[0196] Let's first combine Figure 2A-2I The global collection mode provided in this application and multiple forms of the global collection mode are introduced.

[0197] Global Collection is a management application APP that can collect, organize and share materials. That is, users can use the Global Collection application to collect, organize and share materials. Furthermore, the Global Collection application can be built into a personal learning space, work space, etc. The data retention rule in the Global Collection is permanent retention.

[0198] When using global collections, users can present multiple forms to distinguish and indicate that the electronic device is in different processing stages or performing different processes. For example, the forms of global collections may include hidden state (also called hidden state icon), collection state (also called collection state icon), feedback state (also called feedback state icon), floating state (also called floating state icon) and full-screen state (also called full-screen interface), etc.; the names here are only schematic, for example, the hidden state can also be called minimized state, and the floating state can also be called distributed state, which is not limited in this application. Each form of global collection can also include one or more styles, and the styles can be switched between each other in response to user triggers. See. Figure 2A , which is an example of a global collection in different forms provided in an embodiment of the present application.

[0199] Among them, the hidden state of the global collection is used to indicate that the global collection has been called by the foreground or is running in the background; the collection state is used to indicate the location of the material collection entrance or to indicate that the global collection is in the process of collecting materials; the feedback state is used to indicate the feedback given to the user when collecting materials, telling the user whether the collection is successful; the floating state is used to display the materials collected in the global collection in part of the current display interface; the full-screen state is used to display the materials collected in the global collection in the entire display area. It can be understood that the floating state is equivalent to the display window of the global collection application, and the full-screen state is equivalent to the main interface of the global collection application. The floating state and full-screen state can support users to organize materials.

[0200] Optionally, the shape of the hidden icon of the global favorite can be circular or semicircular, and can be non-transparent or semi-transparent. For example, when the global favorite is in the hidden state, the hidden icon can be presented in a semi-transparent or semi-circular effect adsorbed on the inner side of the screen edge, such as Figure 2A In this case, when icon b is located inside the edge of the screen, it will not affect the user's normal viewing of the displayed content. Icon b is in a semi-transparent state and will not visually attract the user's attention, and will not interfere with the user's normal use.

[0201] It should be noted that the position of the inner side of the screen edge described in the embodiment of the present application can be specifically located on the upper side, lower side, left side or right side of the inner side of the edge, and the present application does not limit this.

[0202] Optionally, the collection state icon of the global collection can be switched to different styles based on different collection scenarios. For example, when collecting text, images and other materials, the collection state style can be switched to Figure 2A When collecting web page content, the collection state style can be switched to Figure 2A The collection state icon c2 shown in FIG. 2; when collecting excerpted videos, the collection state style can be switched to the following Figure 2A The collection state icon c31 and collection state icon c32 shown in FIG; when collecting global annotations, the collection state style can be switched to Figure 2A The collection state icon c41 or the collection state icon c43 and the collection state icon c42 are shown in FIG. The style of the collection state can be specifically set according to the type of collected material, and this application does not limit this.

[0203] It should be noted that the global collection can be switched from the hidden state to the collected state, and vice versa. Different styles between the collected states can also be switched. Optionally, the switching can be an animation effect such as scrolling, telescoping, etc. For example, Figure 2B As shown in the figure, the hidden state of the global collection can be switched from the leftmost figure to the rightmost collection state with a gradually extending animation effect; during the extension process, the hidden state icon b can first switch to a circle, then stretch from the circle to a capsule shape, and gradually display all the texts. This is only an example, and this application is not limited to this.

[0204] Optionally, after the material is collected, the global collection can be switched from the collection state to the feedback state. Figure 2A As shown, the feedback state can be consistent with the style of icons c1, c2, c32, c42, c43, etc. in the collection state, and display the text "Successfully collected" to give feedback to the user in text form that the current material has been successfully collected. Optionally, the graphic in the feedback state can also be switched to a check mark to give feedback in animation form that the current material has been successfully collected.

[0205] Optionally, the floating state of the global collection can move in the display area based on the user's drag operation, and can also change the size based on the user's drag operation. For example, the floating state of the global collection can specifically include two styles, one is a long strip, such as Figure 2A The first suspension state e1 shown in FIG; the other is a card-shaped, such as Figure 2A The second suspended state e2 shown in the figure is different in position and size from the first suspended state e1 in the shape of a strip to the second suspended state e2 in the shape of a card. The first suspended state e1 of the global collection can be displayed as being adsorbed on the inner side of the edge of the display screen to avoid interfering with the currently displayed content. The second suspended state e2 of the global collection can be displayed in the center area of ​​the display screen to facilitate user viewing.

[0206] Optionally, as a possible implementation method, in response to a drag operation on the edge of the first floating state e1, the first floating state e1 can be switched to the second floating state e2; conversely, in response to the user's drag operation on the edge of the second floating state e2, the second floating state e2 can also be switched to the first floating state e1. As another possible implementation method, in response to a two-finger reverse sliding operation on the first floating state e1, the first floating state e1 can be switched to the second floating state e2; conversely, in response to a two-finger pinching operation on the second floating state e2, the second floating state e2 can also be switched to the first floating state e1. In addition, the first floating state e1 can display icons corresponding to one or two columns of materials, and can display at least one icon corresponding to a material. The second floating state e2 can display icons corresponding to two or more columns of materials, but the number of columns and rows is less than that of the full-screen state, and can be set according to needs, and this application does not limit this.

[0207] Optionally, the floating state of the global collection may also include a top bar control and / or a bottom bar control. In response to a click operation on the top bar control, such as Figure 2C As shown, the first suspension state e1 can be switched to Figure 2D The suspended state style with a toolbar is shown; the toolbar of this style may include three controls, namely, a zoom-in control, a zoom-out control, and a close control. The zoom-in control is used to trigger switching from the first suspended state e1 to the full screen state; the zoom-out control is used to trigger switching from the first suspended state e1 to the hidden state b; and the close control is used to trigger exiting the global collection.

[0208] Alternatively, if Figure 2E and Figure 2F As shown, in response to a sliding operation on the top horizontal bar control in the first suspended state e1, the global collection can be switched from the first suspended state e1 to the hidden state b. The direction of the sliding operation can be left or right. Figure 2E Only a swipe to the right is indicated.

[0209] Alternatively, if Figure 2G and Figure 2H As shown, in response to a sliding operation on the bottom horizontal bar control of the first floating state e1, the global collection can be switched from the first floating state e1 to the full screen state f. The direction of the sliding operation can be left or right. Figure 2G Only a swipe to the right is indicated.

[0210] Similarly, the second floating state e2 can also be switched to a floating state style with a toolbar; the second floating state e2 can also include a top horizontal bar control and a bottom horizontal bar control, and the top horizontal bar control and the bottom horizontal bar control of the second floating state e2 can also be slid to switch to a hidden state or a full-screen state accordingly; for the functional introduction of the second floating state e2, please refer to the above description and will not be repeated here.

[0211] It should also be noted that although the global collection will switch between multiple forms during use, the global collection application in each form is at the top of the multiple display layers included in the display interface (or called top), and is not covered or affected by the content displayed below.

[0212] In combination with the above-mentioned global collections and the multiple forms of global collections during use, the cross-terminal interaction method provided by the present application is implemented. Multiple electronic devices based on the same trust ring can execute one or more of the following three processing stages (collection, organization and sharing), and are not limited to the following processes and sequence.

[0213] The following description takes the first device as an example, and the first device may be any electronic device in the same trust ring.

[0214] It is understandable that the materials include various types, and the various types of materials mentioned here may include fragmented materials such as text, graphics, images, tables, annotations, etc., and may also include non-fragmented materials such as web pages, videos, files, etc. The graphics may indicate a partial image captured using a lasso tool, etc.

[0215] The first processing stage is to collect and synchronize materials.

[0216] Function 1. The first device can support the collection of materials through global collection on the local side.

[0217] Function 2. The first device can also support the synchronization of the corresponding icon and device source information of the material to the global collection of the trusted device for display and use after the material is successfully collected by the terminal. The trusted device can be one or more.

[0218] Optionally, a badge can be displayed on any corner of the cross-end synchronized material icon, which is used to indicate that the material is collected on other devices synchronized through the trust ring, rather than collected on the local end. It can be understood that based on the same trust ring, if any device in the ring successfully collects material, all other devices in the ring can synchronize to the icon corresponding to the material.

[0219] Here, the corner mark can be set at any of the upper left corner, lower left corner, upper right corner or lower right corner of the material icon synchronized across terminals, and this application does not limit this.

[0220] Exemplarily, the corner mark may include a device icon, which indicates the source of the device. For example, taking a mobile phone as an example, Figure 2H As shown, for the material icons included in the full-screen state, the corner mark displayed in the upper right corner may include a tablet icon 21, which is used to indicate that the device source corresponding to the material is a tablet; the corner mark may also include a desktop computer icon 22, which is used to indicate that the device source corresponding to the material is a tablet, and the others are similar and will not be repeated here. In addition, the third icon is a group icon, which includes two materials. Therefore, the group icon may include two corner marks, respectively indicating the device sources corresponding to the two materials.

[0221] Exemplarily, the corner mark may also include a trust ring icon, which indicates that the material is synchronized across terminals via a trust ring. Fig.2I As shown, the corner mark displayed in the upper right corner of the cross-end synchronized material icon includes a trust ring icon 23; and / or, the device source can also be displayed in text form under the name of each material icon. Fig.2I As shown, for the group icon including cross-end synchronized materials, a corner mark including a trust ring icon may be displayed in the upper right corner, but the device source may not be displayed below the name. Of course, this is only an example, and it can also be displayed in other forms, which is not limited by this application.

[0222] For example, taking the case where the corner mark of the cross-end synchronized material icon includes the device icon, after the mobile phone successfully collects text material, the material icon corresponding to the collected text can be synchronized to the global collection in the tablet for display. The user can see the material icon corresponding to the text by viewing the global collection application in the tablet; moreover, the top right corner of the icon can also display a corner mark including the mobile phone icon to indicate that the material is synchronized with the mobile phone.

[0223] Function 3. The first device can support cross-end invocation of related services of a trusted device and collect materials through the trusted device. There can be one or more trusted devices.

[0224] Function 4. After the trusted device collects the material, it can also support synchronizing the icon and device source information corresponding to the material to the global collection of the first device and other trusted devices for display.

[0225] For example, the first device can support, in response to the user's operation on the photo option or the video option in the global collection of the first device (the photo option and the video option can also be combined into one photo option), starting the camera application of the second device, calling the camera service, and then using the camera of the second device to take photos or record videos, generate images or video materials and collect them into the global collection application of the second device. It is understandable that the generated images or videos are also saved in the gallery application of the second device at the same time.

[0226] For example, the first device can support, in response to a user's operation on a recording option in the global collection of the first device, starting a recording application of the second device, calling an audio service, and then using the audio service of the second device to record, generate an audio file, and collect it into the global collection application of the second device. It is understandable that the generated audio file is also saved in the recording application of the second device at the same time.

[0227] For another example, the first device can also support, in response to the user's operation on the handwriting option in the global collection of the first device, starting the note application of the second device, calling the handwriting service in the note, and then supporting the handwriting service in the note of the second device for handwriting input, generating a note file and collecting it in the global collection application of the second device. It is understandable that the generated note file is also saved in the note application at the same time.

[0228] In the above three examples, after the second device successfully collects the materials, it can also synchronize the icons and device source information (here from the second device) corresponding to the generated images, videos, audios, and notes to the global collection of the first device for display. In addition, when the same trust ring also includes a third device, the material icons and device source information collected by the first device calling the second device service across terminals can also be synchronized to the global collection of the third device for display.

[0229] Function 5. After the first material is successfully collected, the first device can support a cross-end pop-up notification on the trusted device to prompt the user on the trusted device that the first device has collected new materials through global collections.

[0230] The second processing stage is to organize the materials and synchronize the results.

[0231] Function 1. When organizing materials, the first device can support grouping of multiple materials.

[0232] Among them, the present application can support grouping of multiple materials of the same type, and can also support grouping of multiple materials of different types. The present application can also support grouping of multiple individual materials, forming a new group for one or more individual materials and one or more groups of materials, and forming a new group for multiple groups of materials, etc. It should be noted that grouping of materials can be understood as placing multiple materials in a folder. When the already grouped materials are organized into a new group, the new group will include the individual materials in the original group.

[0233] Function 2. When organizing materials, the first device can support moving one or more materials at a time.

[0234] It can be understood that the movement can indicate movement within the plane, or movement in the depth of the display path. For example, the movement can indicate the translation of the display position of a material in the display interface, from top to bottom, from left to right, etc.; or, the movement can also indicate one or more materials moving to a group of materials, or one or more materials grouped, both of which are moving from the primary interface to the secondary interface. The primary interface can indicate the full-screen state or floating state of the global collection, and the secondary interface can indicate the interface corresponding to a group of materials or the interface corresponding to opening a folder.

[0235] Function 3. When organizing materials, the first device can support renaming of a material or a group of materials at a time.

[0236] Function 4. When organizing materials, the first device can support deleting one or a group of materials.

[0237] It should be noted that for the above functions 1 to 4, the first device responds to user operations and performs grouping, moving, renaming and deleting operations on the collected materials, and will synchronize the results of the organization to the trusted device after organization. For example, if the materials are grouped on the first device, all trusted devices based on the same trust ring will synchronize the material grouping results; if the materials are deleted on the first device, all trusted devices based on the same trust ring will delete the materials synchronously.

[0238] With respect to the above functions 1 to 4, the material on the first device may include material collected by the first device itself and / or material synchronized to a trusted device.

[0239] The above only schematically shows the four functions of organizing materials. Of course, other functions may also be included, such as filtering materials, etc. This application does not limit the functions of the organizing stage. The organizing method provided by this application can achieve the purpose of quickly classifying and reorganizing the collected materials by combining the global collection application, so that the display method of the collected materials meets the personal needs of users and improves the user experience.

[0240] It should be noted that, since the global collection can partially or fully display the collected materials in the global collection when it is in a suspended state or full-screen state, the organization of the collected materials mainly involves the suspended state and full-screen state of the global collection application.

[0241] The third processing stage: sharing materials.

[0242] Optionally, the sharing mentioned here may include material transfer or cross-application use of materials, as well as material sharing, etc.

[0243] When using the material, the first device can respond to the user's drag operation to drag the material in the global collection to a certain application, and then respond to the release operation to import the material into the application of the first device. The material can include local material or cross-end synchronized material.

[0244] Here, if the synchronized material has been downloaded to the first device, the material can be directly imported into the application of the first device; if the synchronized material has not been downloaded, it can be downloaded first and then imported during the process of importing it into the application of the first device.

[0245] It is understandable that the sharing functions involved above are implemented on each device separately; based on devices in the same trust circle, new materials included in the global collection have been automatically synchronized, so there is no need to share manually across devices.

[0246] It should be noted that in addition to the first processing stage to the third processing stage, the electronic device can also implement other processing. Based on the first processing stage to the third processing stage, the electronic device can more conveniently collect, organize and share materials. For the relevant content in the scene of collecting, organizing and sharing materials, please refer to the following description, which will not be repeated here.

[0247] The first processing stage (collecting materials) is introduced in detail below.

[0248] 1. The first device collects local material and synchronization material (such as Figure 3A-3J (shown)

[0249] Taking a news application as an example, the first device can detect a user operation on the news application. In response to the user operation, the first device can start the news application and display the main interface of the news application (i.e., the interface displayed by default after startup). The main interface of the news application can include thumbnails of multiple news messages. The thumbnails of the multiple news messages can be used to trigger the first device to display the specific content of the news message. The first device can detect a user operation on any thumbnail. In response to the user operation, the first device displays the display interface of the news message corresponding to the corresponding thumbnail.

[0250] Optionally, the first device may detect a user operation on a section of text in the display interface, and in response to the user operation, a portion of the text may be in a text selected state; then, the first device may detect a user operation on the portion of text in the text selected state. In response to the user operation, the first device may enter a global collection mode.

[0251] Optionally, when part of the text is in a text selected state or a non-text selected state, the text background color may be different colors to distinguish. For example, when part of the text is in a non-text selected state, the text background color may be white, and when it is in a text selected state, the text background color may be gray.

[0252] It can be understood that the user operations mentioned in the embodiments of the present application may include but are not limited to touch (for example, click, long press, etc.), voice control, gestures and other operations, and the present application does not limit this.

[0253] Take the first device as a mobile phone as an example, see Figure 3A The display interface 301 shown is a web page of a news message in a news application. The mobile phone can detect a selection operation (such as a left or right sliding operation after long pressing) for a section of text in the display interface 301. In response to the selection operation, part of the text can be in a text selected state. For the convenience of description, the application can record the part of the text in the text selected state as text a1 (such as Figure 3A Text with a gray background).

[0254] The mobile phone can detect the long press operation on the text a1, and in response to the long press operation, the mobile phone can display the following Figure 3B The display interface 302 shown. Compared with the display interface 301, the display interface 302 displays a hidden icon b of the global collection on the right side of the interface; the icon b is used to remind the user that the mobile phone has entered the global collection mode triggered by the long press operation on the text a1, and the text a1 in the text selected state can be collected by the global collection application.

[0255] Optionally, when the mobile phone detects a long press operation on a portion of the text in a selected state, a move icon a2 (such as FIG. 3B to FIG. 3D The moving icon a2 is used to prompt the user that the part of the text in the selected state can be moved in any direction in response to the user's dragging operation in a plane parallel to the display interface.

[0256] See also Figure 3CIn the display interface 303 shown, the mobile phone can detect a drag operation on the text a1. In response to the drag operation, the text a1 in the text selected state can be split and displayed as two texts (for the convenience of description, they can be called foreground text and background text). The foreground text is the same as the original text a1 in the text selected state, or the font is relatively reduced, or the font is relatively reduced and a border, background color, etc. are added; the background text is the same as the original text a1, or the background text can be switched to a gray font. In addition, the foreground text can be moved in the display interface 303 with the user's drag operation, and the background text does not move with the user's drag operation.

[0257] At the same time, if Figure 3C As shown in , in response to the drag operation, the global favorite application is called in the foreground, and the hidden state icon b of the global favorite gradually stretches from the right side to the left side of the display interface and switches to the collection state icon c of the global favorite.

[0258] See also Figure 3D In the display interface 304 shown, the mobile phone can also detect the user operation on the foreground text corresponding to the text a1. In response to the user operation, the foreground text can move to the position of the collection icon c of the global favorites along with the user operation and then disappear.

[0259] Optionally, the user operations mentioned in this embodiment may include but are not limited to dragging, throwing and other operations, which are not limited in this application. The dragging direction and the throwing direction are both toward the location of the global collection icon c on the display interface. In addition, the position where the drag stops overlaps with the position of the global collection icon c. If there is no overlap, its content may not be collected into the global collection application.

[0260] Optionally, in response to a user operation, when the foreground text moves to the location of the collection state icon c of the global collection with the user operation, since the text belongs to the type of material that can be collected in the global collection, a feedback badge can also be displayed in the upper right corner of the foreground text. Figure 3D As shown, a circle containing a "+" may be displayed in the upper right corner of the foreground text to indicate to the user that the foreground text is allowed to be collected into the global favorites app at this time. It is understandable that if it is not collectible, another feedback badge may be displayed in the upper right corner, for example, a circle containing a "\" may be displayed in the upper right corner of the foreground text to indicate to the user that it is not allowed to be collected into the global favorites app at this time.

[0261] Optionally, in response to user operations, the size of the foreground text can also change from large to small in the process of moving to the location of the global favorite collection icon c, forming a gradually disappearing visual effect. Correspondingly, the global favorite collection icon c can show dynamic feedback of enlarging and then shrinking in this process, and the interaction between the two forms a visual effect that the collection icon c absorbs the foreground text.

[0262] Optionally, as a possible implementation, combining Figure 3D The display interface 304 and Figure 3F In the display interface 306 shown in the figure, after the mobile phone collects the text a1 into the global collection application, the collection is completed, the background text is switched to black font, and the collection state icon c of the global collection can be switched to the hidden state icon b; as another possible implementation method, combined with Figure 3D The display interface 304 shown, Figure 3E The display interface 305 and Figure 3F In the display interface 306 shown, after the mobile phone collects the text a1 into the global collection application, the collection is completed, the background text is switched to black font, and the collection state icon c of the global collection can be switched to the feedback state icon d first, and then switched from the feedback state icon d to the hidden state icon b. The basic form of the feedback state icon d can remain unchanged compared to the collection state icon c, and the text on the icon can be displayed as "Collected successfully", and a check mark can also be displayed on the icon to feedback to the user that the text a1 has been collected into the global collection.

[0263] Optionally, after the collection is successful, the global collection application may continue to be displayed on the screen in a hidden state to indicate to the user that the global collection application is still running in the background, and the user may collect any material into the global collection at any time.

[0264] It can be understood that the above is an example of starting global collection to collect texts on the local side when the mobile phone is used as the first device.

[0265] When the collection is successful, the global collection application is displayed on the screen in a hidden state, such as Figure 3G In the display interface 306 shown, the mobile phone can detect a user operation on a hidden state icon; in response to the user operation, the global favorite application can be switched from the hidden state to the suspended state for display, such as Figure 3H In the display interface 307 shown, the global collection floating state may include an icon 3071 corresponding to the newly collected material in the above example, such as "Text 01". Here, since the mobile phone collects the material on the local device, there is no corner mark indicating the device source in the upper right corner of the icon 3071.

[0266] Based on the above example, the first device can support the synchronization of the corresponding icon of the material to the global collection of the trusted device for display after the material is successfully collected by the local end. The trusted device can be one or more. It is understandable that when there are multiple trusted devices based on the same trust ring, they can be synchronized to the global collection of multiple devices for display.

[0267] For example, after the mobile phone collects the local material "Text 01" into the global collection, based on the same trust ring, it can send data such as the icon and device source information corresponding to the material to other devices in the ring, such as a tablet. After the tablet receives the data, it can synchronously display the icon and device source (in the form of a corner mark) corresponding to the new material in the global collection of the tablet.

[0268] For tablets that are trusted devices, the following information can be displayed: Fig. 3I When the tablet detects a user operation for hiding the global favorites displayed on the desktop, it can display the following Figure 3J The display interface 309 shown in the figure includes a new material icon 3091 synchronized to the floating state, and the icon 3091 is used to indicate the text 01 collected by the mobile phone. Here, the difference between the icon 3091 and the icon 3071 is that the upper right corner of the icon 3091 also displays a corner mark indicating that it comes from the mobile phone. The icons synchronized on other trusted devices are the same as the icon 3091, and will not be repeated here.

[0269] It can be understood that the above is an example in which the first device collects materials on the local end and synchronizes the icons of the successfully collected materials to the global collection of the trusted device.

[0270] It should be noted that when a tablet, laptop, etc. is used as the first device, the process of starting global collection to collect text on the local end, and the process of viewing after successful collection are similar to the above. For details, please refer to the above Figure 3A-3H The introduction of , I will not go into details here.

[0271] For example, Figure 3J The icon 3092 displayed in the suspended state in the display interface 309 shown is used to indicate the material collected by the tablet. Therefore, there is no corner mark when it is displayed on the tablet; when it is synchronized to the mobile phone end as a trusted device of the tablet, such as Figure 3H In the display interface 307 shown, the same icon will be displayed in the suspended state and a corner mark indicating that it comes from the tablet will be superimposed on the icon, such as Figure 3H Icon 3072 shown.

[0272] Understandable, combined Figure 3HThe icon 3071 shown in the figure, the first device (such as a mobile phone) can also detect a click operation on the icon 3071 later; in response to the click operation, the first device can display the text material indicated by the icon 3071. Optionally, as an implementation method, in response to the click operation on the icon 3071, the first device can display a full-screen preview interface corresponding to the text material, or, as another implementation method, the first device can also display the text material in the form of a floating window. Thus, the user can view the text material that has been collected in the global collection application by clicking on the icon 3071. Here, the floating window is pinned to the top.

[0273] Similarly, combined Figure 3J The tablet computer can also detect a click operation on the icon 3091, and in response to the click operation, the tablet computer can download and display the text material indicated by the icon 3091. Optionally, as an implementation, in response to the click operation on the icon 3091, the tablet computer can display a full-screen preview interface corresponding to the text material, or, as another implementation, the tablet computer can also display the text material in a floating window. Thus, the user can download and view the text material collected from the first device to the global collection application by clicking the icon 3091.

[0274] It should be noted that the above is to display the relevant introduction of the materials collected in the global collection in the form of a full-screen preview interface or a floating window in response to the triggering of the icon. Of course, it can also be displayed in other ways. For example, when the first device (such as a folding screen mobile phone) is in a multi-screen display state, in response to the triggering of the icon, the floating icon can continue to be displayed on one of the multiple screens, and the text material can be displayed in full screen on another screen.

[0275] It is understandable that the full-screen preview interface or floating window also includes a return control. When the first device or the trusted device detects a click operation on the return control, the full-screen preview interface or floating window can be closed, and then return to the display interface when the icon is triggered.

[0276] 2. The first device calls the trusted device to collect materials across devices (such as Figure 4A-Figure 4I As shown, Figure 5A-5H As shown and Figure 6A-6D (shown)

[0277] 1. Trigger entry on the first device.

[0278] Take the first device as a desktop computer as an example, see Figure 4A and Figure 4B As shown, the first device runs a global favorites application, and the floating state of the global favorites can be displayed on the display interface 401, such as Figure 2A The suspended state e1 is shown.

[0279] Alternatively, if Figure 4A As shown, the floating upper right corner of the global favorite may include three dots for indicating a "more" control. The first device may detect a user operation on the "more" control, and the "more" control enables more functions. In response to the user operation, such as Figure 4B In the display interface 402 shown, the first device can display a card 4021 below the "More" control; the card 4021 includes but is not limited to "Create Desktop Shortcut", "Clipboard" and "Settings" options.

[0280] Alternatively, if Figure 4B As shown, when the first device forms a trust ring with the second device (such as a mobile phone) and the third device (such as a tablet), the card 4021 may also include multiple callable service options involved in the second device, such as "photographing", "video recording", "recording" and "handwriting". Similarly, the card 4021 may also include multiple callable service options involved in the third device, and the callable service options of the third device are the same as those of the second device. It is understandable that when the first device forms a trust ring with multiple devices, the card 4021 may display the service options involved in the multiple devices respectively, and the service options corresponding to different devices may be the same or different, and this application does not limit this.

[0281] Optionally, the first device may also display a full-screen state of the global collection, and the upper right corner of the full-screen state of the global collection may also include three dots for indicating a "more" control, and the "more" control may enable more functions; the first device may detect user operations on the "more" control, and in response to the user operations, the first device may also display the above-mentioned card 4021. For details, please refer to the above description and will not be repeated here.

[0282] 2. Call the service of the second device across the first device and exit the service.

[0283] Based on the above card 4021, the first device can detect a user operation on an option related to the second device or the third device on the card 4021; in response to the user operation, the first device can cross-end launch a certain application of the second device or the third device, and at the same time, a multi-screen collaborative interface can be displayed on the first device, and the multi-screen collaborative interface includes the display interface related to the application; the first device controls the second device or the third device to collect materials in response to the user operation on the display interface related to the application. In addition, the multi-screen collaborative interface also includes an exit control, and the first device can exit the control of the second device or the third device in response to the user operation on the exit control.

[0284] The following uses three embodiments to illustrate how a first device can call the photo taking, recording, and handwriting services of a second device across terminals.

[0285] Example 1 - Taking photos or recording videos across devices

[0286] like Figure 4C As shown, taking a mobile phone as the second device, the first device can detect a user operation on the photo option corresponding to the mobile phone in card 4021; in response to the user operation, the first device can send an instruction to start the camera application on the mobile phone, instructing to call the camera service on the mobile phone to take a photo. At the same time, in response to the user operation, the first device can start the smart interconnection function of the trust ring and switch from card 4021 to the multi-screen collaborative interface for display.

[0287] like Figure 4D As shown, when the mobile phone receives the instruction sent by the first device to start the camera application to take a photo, the camera application can be started and the desktop or other display interface is switched to the photo taking interface in the camera application, such as Figure 4E As shown, the photo interface may include a preview display area, photo controls, thumbnail controls, flip lens controls, multiple photo mode options, etc. The preview display area is used to display the shooting scene image captured by the mobile phone's camera in real time; the photo control is used to trigger photo taking, the thumbnail control is used to trigger the display of the image generated by the last photo, the flip lens control is used to trigger the switching of the front and rear cameras, and the multiple photo modes are used to trigger the generation of images or videos using different algorithms, etc.

[0288] like Figure 4F As shown, for the first device, in response to the user operation on the photo option corresponding to the mobile phone in the card 4021, the first device can switch from the card 4021 to the multi-screen collaborative interface 4022 for display, and the multi-screen collaborative interface 4022 may include the photo interface involved in the camera application in the mobile phone. Based on the multi-screen collaborative interface 4022, the user operates on the first device (such as clicking on the photo control), so that the mobile phone can be controlled across terminals to take pictures. The image generated by this photo will be automatically collected in the global collection application of the mobile phone after the picture is output. It can be understood that the image generated by the photo will also be saved in the gallery of the mobile phone at the same time.

[0289] Optionally, when the mobile phone receives an instruction sent by the first device to start the camera application to take a photo, even if the mobile phone is in a locked screen state, the camera application can be started and the lock screen interface can be switched to a photo taking interface in the camera application.

[0290] Optionally, the first device can continue to detect user operations on the photo controls in the photo interface included in the multi-screen collaborative interface 4022. In response to the user operations, the first device can control the mobile phone to take multiple photos; the images generated by each photo will be automatically collected into the global collection application of the mobile phone after the image is taken, and will also be saved in the mobile phone's gallery.

[0291] Optionally, for the first device, when switching from card 4021 to multi-screen collaborative interface 4022, the global collection floating state on the first device can be retracted and hidden; or, it can continue to be displayed.

[0292] Alternatively, if Figure 4G As shown, the upper right corner of the multi-screen collaboration interface displayed on the first device also includes an exit control, and the first device can detect a user operation for the exit control; in response to the user operation, the first device exits the control of the mobile phone, and the multi-screen collaboration interface 4022 is closed. Accordingly, after the mobile phone releases the control, the display interface before being controlled can continue to be displayed.

[0293] Alternatively, if Figure 4H As shown, in response to the user operation for exiting the control, the first device may also display a prompt card 4023, which may display the text "The camera function has been exited from a certain mobile phone" to prompt the user that the first device has exited control of the mobile phone.

[0294] Optionally, after taking a photo, in response to a user operation to exit the control, while the multi-screen collaboration interface is closed, the first device may display the hidden state or suspended state of the global collection. In response to an operation on the hidden state, when the hidden state is switched to the suspended state or when the first device displays the suspended state of the global collection, such as Figure 4H As shown, the suspended state may include image icons collected by the synchronized mobile phone, such as icon 4024. Since the material comes from the mobile phone, when displayed on the first device, the upper right corner of the icon may include a badge, indicating that the device source information is a mobile phone. It can be understood that the global collection in the mobile phone also includes the image icon but does not include the badge in the upper right corner.

[0295] Optionally, in order to prompt that the material collected by the cross-terminal call mobile phone has been synchronized to the first device, the first device may also display a card next to the synchronized material icon to indicate the location of the new material or the viewing entrance. Figure 4H As shown, the first device may display a card 4025 including the text "Cross-end photo taking is successful, click here to view new material" next to the synchronized image icon 4024. Subsequently, in response to a click operation on the icon 4024, the first device may display a full-screen preview interface corresponding to the image in the global collection; or, the image may be displayed in a floating window, which is not limited in the present application.

[0296] It can be understood that the above is an example of the first device cross-terminal calling the camera service on the mobile phone to take pictures, realizing image collection, exiting the cross-terminal call, and displaying the synchronized icon on the first device. The process of the first device controlling other trusted devices to take pictures can be referred to the above description, which will not be repeated here.

[0297] In addition, the first device calls the camera service on the mobile phone across terminals to record video, and the process of implementing video collection is similar to the above. Please refer to the above description for details, which will not be repeated here.

[0298] Example 2 - Cross-end recording

[0299] like Figure 5A As shown, taking the mobile phone as the second device, the first device can detect the user operation on the recording option corresponding to the mobile phone in card 4021; in response to the user operation, the first device can send an instruction to start the recording application on the mobile phone, instructing to call the audio service on the mobile phone for recording. At the same time, in response to the user operation, the first device can start the smart interconnection function of the trust ring and switch from card 4021 to the multi-screen collaborative interface for display.

[0300] like Figure 5B As shown, when the mobile phone receives the instruction sent by the first device to start the recording application for recording, the recording application can be started and the recording interface in the recording application can be switched from the desktop or other display interface, such as Figure 5C As shown, the recording interface may include a recording area, a recording control, a mark control, a pause control, etc. The recording area is used to display the waveform and timeline when the recording application of the mobile phone records sound in real time; the recording control is used to trigger recording; the mark control is used to trigger marking of audio information at a certain moment; the pause control is used to trigger pause recording, and after pausing, the pause control can be switched to a continue control, and the continue control is used to trigger continued recording.

[0301] like Figure 5D As shown, in response to the user operation on the recording option corresponding to the mobile phone in card 4021, the display content on the first device can be switched from card 4021 to a multi-screen collaborative interface 4024, which may include a recording interface involved in the recording application in the mobile phone. Based on the multi-screen collaborative interface 4024, the user can operate on the first device (such as clicking on the recording control) to control the mobile phone to record across terminals. The audio file generated by this recording will be automatically collected in the global collection application of the mobile phone after the recording is completed. It can be understood that the audio file generated by the recording will also be saved in the recording application at the same time.

[0302] Optionally, when the mobile phone receives an instruction from the first device to start a recording application for recording, if the mobile phone is in a lock screen state, then when the display content on the first device switches from card 4021 to the multi-screen collaboration interface, the multi-screen collaboration interface can first display a lock screen prompt interface including a prompt message, prompting the user that the mobile phone is in a lock screen state and needs to be unlocked on the mobile phone; when the mobile phone is unlocked, the display content on the first device switches from the lock screen prompt interface to the recording interface.

[0303] For example, Figure 5E As shown, if the mobile phone is in a locked state, the multi-screen collaboration interface displayed on the first device may display a lock screen prompt interface 4025 including a prompt "locked, need to be unlocked on the mobile phone" to prompt the user to unlock the phone. Fig. 5F As shown, a mobile phone in the locked screen state can also display a prompt, such as "The computer is calling the recording function. If you agree, please unlock it" to prompt the user to unlock it. Figure 5G and Figure 5H As shown, when the mobile phone is successfully unlocked in response to the user's unlocking operation, the recording application can be started and the recording interface can be displayed. The first device side will also switch from the lock screen prompt interface 4025 to the following in response to the unlocking operation. Figure 5D The multi-screen collaboration interface shown includes a recording interface.

[0304] Optionally, the first device can continue to detect user operations on the recording controls in the recording interface included in the multi-screen collaborative interface 4024. In response to the user operations, the first device can control the mobile phone to perform multiple recordings; the audio files generated by each recording will be automatically collected into the global collection application of the mobile phone after the recording is completed, and will also be saved in the recording application of the mobile phone.

[0305] Optionally, for the first device, when switching from card 4021 to multi-screen collaborative interface 4024, the global collection floating state on the first device can be retracted and hidden; or, it can continue to be displayed.

[0306] Alternatively, if Figure 5D As shown, the upper right corner of the multi-screen collaboration interface 4024 displayed on the first device also includes an exit control, and the first device can detect a user operation for the exit control; in response to the user operation, the first device exits the control of the mobile phone, and the multi-screen collaboration interface 4024 is closed. Accordingly, after the mobile phone releases the control, the display interface before being controlled can continue to be displayed.

[0307] Optionally, in response to a user operation to exit the control, the first device may further display a prompt card, which may display the text "The recording function has been exited from a certain mobile phone" to prompt the user that the first device has exited control of the mobile phone.

[0308] Optionally, after recording, in response to a user operation on an exit control, while the multi-screen collaborative interface is closed, the first device may display the hidden state or suspended state of the global collection. In response to an operation on the hidden state, when the hidden state is switched to the suspended state or when the first device displays the suspended state of the global collection, the suspended state may include an audio icon collected by the synchronized mobile phone. Since the material comes from the mobile phone, when it is displayed on the first device, a superscript may be included in the upper right corner of the icon, indicating that the device source information is a mobile phone. It can be understood that the global collection in the mobile phone also includes the audio icon but does not include the superscript in the upper right corner.

[0309] Optionally, in order to prompt that the material collected by the cross-end call mobile phone has been synchronized to the first device, the first device can also display a card next to the synchronized material icon to indicate the location of the new material or the viewing entrance. For example, the first device can display a card including the text "Cross-end recording has been successful, click here to view new material" next to the synchronized audio icon. Subsequently, in response to a click operation on the icon, the first device can display the full-screen playback interface corresponding to the audio in the global collection.

[0310] It can be understood that the above is an example of the first device calling the audio service on the mobile phone across terminals to record audio, realize audio collection, exit the cross-terminal call, and display the synchronized icon on the first device. The process of the first device controlling other trusted devices to record audio can refer to the above description, which will not be repeated here.

[0311] Example 3 - Cross-end handwriting

[0312] like Fig. 6A As shown, taking the mobile phone as the second device, the first device can detect the user operation on the handwriting option corresponding to the mobile phone in the card 4021; in response to the user operation, the first device can send an instruction to start the note application on the mobile phone, instructing to call the handwriting service on the mobile phone for handwriting input. At the same time, in response to the user operation, the first device can start the smart interconnection function of the trust ring and switch from card 4021 to the multi-screen collaborative interface for display.

[0313] like Figure 6B As shown, when the mobile phone receives the instruction sent by the first device to start the note application for handwriting input, the note application can be started and the desktop or other display interface can be switched to the handwriting input interface in the note application, such as Figure 6C As shown, the handwriting input interface may include a title, current time, a handwriting input area, a handwriting toolbar, etc. The handwriting input area is used to display handwriting traces collected by the mobile phone in real time, and the handwriting toolbar is used to trigger the selection of handwriting tools, colors, etc.

[0314] like Fig.6DAs shown, in response to the user operation on the handwriting option corresponding to the mobile phone in card 4021, the display content on the first device can be switched from card 4021 to a multi-screen collaborative interface 4026, which may include a handwriting input interface involved in the note application in the mobile phone. Based on the multi-screen collaborative interface 4026, the user can perform input operations on the first device (such as writing in the handwriting input area), so that the mobile phone can be controlled across terminals for handwriting input; after the user completes the handwriting input on the first device, the completion control in the upper right corner of the handwriting input interface can be clicked; in response to the click operation, the handwriting input is completed. The notes generated by this handwriting input will be automatically collected into the global collection application of the mobile phone after the input is completed. It can be understood that the notes generated by the handwriting input will also be saved in the note application at the same time.

[0315] Optionally, when the mobile phone receives an instruction from the first device to start the note application for handwriting input, if the mobile phone is in a lock screen state, then when the display content of the first device switches from card 4021 to the multi-screen collaboration interface, the multi-screen collaboration joint interface can first display a lock screen prompt interface including a prompt, prompting the user that the mobile phone is in a lock screen state and needs to be unlocked on the mobile phone; when the mobile phone is unlocked, the display content on the first device switches from the lock screen prompt interface to the handwriting input interface.

[0316] Here, we will introduce the lock screen prompt interface and unlocking process. FIG. 5E to FIG. 5H The introduction is similar to that of , and you can refer to the above content for details, so I will not go into details here.

[0317] Optionally, the first device can control the mobile phone to perform multiple handwriting inputs; the notes generated after each handwriting input is completed will be automatically collected into the global collection application of the mobile phone after the input is completed, and will also be saved in the note application of the mobile phone.

[0318] Optionally, for the first device, when switching from card 4021 to multi-screen collaborative interface 4026, the global collection floating state on the first device can be retracted and hidden; or, it can continue to be displayed.

[0319] Alternatively, if Fig.6D As shown, the upper right corner of the multi-screen collaboration interface 4026 displayed on the first device also includes an exit control, and the first device can detect a user operation for the exit control; in response to the user operation, the first device exits the control of the mobile phone, and the multi-screen collaboration interface 4026 is closed. Accordingly, after the mobile phone releases the control, the display interface before being controlled can continue to be displayed.

[0320] Optionally, in response to a user operation to exit the control, the first device may further display a prompt card, which may display the text "Handwriting function has been exited from a certain mobile phone" to prompt the user that the first device has exited control of the mobile phone.

[0321] Optionally, after the handwriting input is completed, in response to the user operation for exiting the control, the first device can display the hidden state or suspended state of the global collection while the multi-screen collaborative interface is closed. In response to the operation on the hidden state, when the hidden state is switched to the suspended state or when the first device displays the suspended state of the global collection, the suspended state may include the note icons collected by the synchronized mobile phone. Since the material comes from the mobile phone, when it is displayed on the first device, the upper right corner of the icon may include a superscript, indicating that the device source information is a mobile phone. It can be understood that the global collection in the mobile phone also includes the note icon but does not include the superscript in the upper right corner.

[0322] Optionally, in order to prompt that the materials collected by the cross-terminal call mobile phone have been synchronized to the first device, the first device can also display a card next to the synchronized material icon to indicate the location of the new material or the viewing entrance. For example, the first device can display a card including the text "Cross-terminal handwriting has been successful, click here to view new materials" next to the synchronized note icon. Subsequently, in response to a click operation on the icon, the first device can display the full-screen display interface corresponding to the note in the global collection.

[0323] It can be understood that the above is an example of the first device cross-terminal calling the handwriting service on the mobile phone to perform handwriting input, realize note collection, exit cross-terminal calling, display the synchronized icon on the first device, etc. The process of the first device controlling other trusted devices to perform handwriting can refer to the above description, which will not be repeated here.

[0324] Understandable, for Figure 3J The icon 3091 shown, Figure 4H As shown in the icon 4024, since the application synchronizes all the material icons, when the user needs to view or use the material on the current device later, the icon will respond to the user's operation and download the material to the current device before displaying it. Optionally, during the download process, if the material data is relatively small, there will be no feedback and the user will not perceive it; if the material data is relatively large, the download progress can be fed back to prompt the user the download start time and download completion time.

[0325] For example, for Figure 4H Icon 4024 shown, such as Fig. 4IAs shown, if the image data is relatively large, the first device can trigger the download of the image data to the first device after detecting the click operation on icon 4024, and then display the full-screen preview interface corresponding to the image in the global collection. Here, icon 4024 can add a mask when downloading starts, and can remove the mask to restore the original display state after downloading is completed. Thus, the user can be prompted with the feedback of the download start time and download completion time. It is understandable that the feedback during the download process can also be provided in other ways, such as including a download progress bar, etc., which is not limited in this application.

[0326] It should be noted that the method provided in the present application can combine the global collection function with the trust ring. Two devices based on the same trust ring can realize cross-end service calls, and then realize cross-end material collection and material synchronization, which expands the material collection scenario and facilitates cross-end operations when users cannot directly operate a certain device, thereby improving the user experience.

[0327] In an embodiment of the present application, whether the first device collects materials on the local end and synchronizes the icons of the successfully collected materials to the global collection of the trusted device, or calls the service on the trusted device across ends to perform cross-end material collection and material synchronization, for the successfully collected materials, in addition to saving them on the source device, the icons are synchronized first when synchronizing on other devices, and the amount of data transmitted is smaller, thereby improving the efficiency of synchronization and thus improving the user experience.

[0328] 3. Cross-terminal notification will be made after the first material collection.

[0329] Optionally, the first device may support pop-up notification on the trusted device across devices after the local end successfully collects the material, so as to remind the user that the global collection on the first device has collected new material.

[0330] For example, in combination Figure 3A-3H As shown, when a mobile phone is used as the first device, global collection is started to collect text materials on the local end for the first time; after the collection is successful, the global collection may include icons corresponding to the collected text materials. Figure 3H As shown in , the floating state of the global collection may include an icon 3071 named "Text 01".

[0331] In combination with this example, if the tablet is a trusted device of the mobile phone, when the text is successfully collected on the mobile phone, the mobile phone can send a notification prompt instruction to the tablet; after receiving the notification prompt instruction, the tablet can pop up a notification card to remind the user that a new material "Text 01" has been collected in the global collection on the mobile phone.

[0332] If the tablet has not started the global collection, then at this time, the tablet can start the global collection application after receiving the notification prompt instruction, such as Fig. 7A and Figure 7B As shown in the figure, the global collection is first displayed in a hidden state, then slides from the hidden state to the left, and gradually displays as a capsule state icon; in this process, Figure 7C As shown, a notification card 701 may appear below the capsule status icon, and the card 701 may display a prompt “New materials have been collected on the mobile phone, click here to view and use them”.

[0333] After the card 701 appears, it can automatically disappear after a preset time, or a button including the text "got it" can be displayed on the card 701, and then, Fig.7D As shown, the tablet can detect user operations on buttons; Fig. 7E and Figure 7F As shown, in response to the user operation, the card 701 may disappear, and the display content on the tablet may be switched from the global favorite capsule state icon to the hidden state icon.

[0334] If the tablet has run global collections, the global collections are displayed in hidden, floating or full-screen state on the display interface. At this time, when the hidden state is displayed, you can switch from the hidden state to the pill state icon, and then display the notification card. For details, please refer to the above Figure 7A-7F The introduction of the above is not described here; when the floating state or full screen state is presented, the notification card can be displayed in the full screen display area, in the floating display area or next to it. It can be understood that after the notification card disappears, the tablet continues to display the display interface before the notification card appears.

[0335] Optionally, the first device may also support pop-up notifications on other trusted devices after successfully collecting materials for the first time across devices using a certain trusted device to remind the user that new materials have been collected in the global collection on the certain trusted device.

[0336] For example, in combination Figure 4A-4H As shown in FIG. 1 , when a desktop computer is used as a first device and a mobile phone is used as a trusted device (second device), the desktop computer uses the mobile phone to take photos across devices. After the mobile phone successfully collects images, an image named "Moon.jpg" is collected for the first time in the global collection of the mobile phone. At this time, the desktop computer can also be synchronized to the icon corresponding to the image, such as Figure 4H Icon 4024 shown in .

[0337] In combination with this example, if the tablet is another trusted device (third device) of the desktop computer, after the image material is successfully collected on the mobile phone, the mobile phone can feedback a message of successful collection to the desktop computer, and the desktop computer can send a notification prompt instruction to the tablet based on the feedback message, or the mobile phone can directly send a notification prompt instruction to the tablet; after receiving the notification prompt instruction, the tablet can pop up a notification card to remind the user that a new material "Moon.jpg" has been collected in the global collection on the mobile phone.

[0338] It can be understood that no matter whether the first device calls the service to collect materials on the local side or across sides, as long as it is the first time to successfully collect the materials, it can send a notification prompt instruction to the trusted devices under the same trust ring or other trusted devices except the called service. The device that receives the notification prompt instruction can pop up a notification card to prompt the user that new materials have been collected on the first device or the trusted device of the called service.

[0339] It can be understood that the above is an example of cross-end notification after the first material collection.

[0340] It should be noted that the method provided by this application can only provide a cross-end prompt after the first successful collection of materials, or can also provide a prompt after each successful collection of materials, or can never provide a prompt. The specific settings can be made as needed, and this application does not limit this. When a cross-end prompt is only provided after the first successful collection of materials, the main purpose is to help users learn the function that the global collection application can synchronize materials across ends based on the same trust ring.

[0341] 4. Prompt when opening a global collection for the first time on a trusted device.

[0342] Optionally, after the first device supports the function of collecting materials locally, when the user opens the global collection application on the trusted device for the first time, the materials collected by the first device are synchronized, and after synchronization, the user is prompted that the material comes from the first device or from other devices.

[0343] For example, in combination Figure 3A-3H As shown, when a mobile phone is used as the first device, the global collection is started to collect text materials on the local end; after the collection is successful, the global collection may include icons corresponding to the collected text materials. Figure 3H As shown, the floating state of the global collection may include an icon 3071 named "Text 01".

[0344] Assemble the example, such as Figure 7G-Figure 7IAs shown, if the tablet is a trusted device of the mobile phone, when the user triggers the launch of the global collection application on the tablet for the first time, in order to remind the user that the global collection has the function of synchronizing materials across terminals, the tablet responds to the user's trigger and displays the floating state or full-screen state of the global collection. It first synchronizes the materials across terminals, and then displays a prompt card under the icon of the material synchronized across terminals. The prompt card may include the text "Content from your other devices" to remind the user that the material is synchronized with the mobile phone.

[0345] Alternatively, if Figure 7H As shown, when synchronizing materials across terminals, the text "Synchronizing" can be displayed before the synchronization result appears to remind the user that cross-terminal synchronization is in progress.

[0346] It can be understood that the above is an example of a prompt when the global collection is opened for the first time on a trusted device after the local end collects the materials.

[0347] It should be noted that the method provided in this application can prompt only one cross-end synchronized material when the trusted device opens the floating state or full-screen state of the global collection for the first time, or can also prompt for multiple cross-end synchronized materials separately. Alternatively, it can also prompt for each newly synchronized cross-end material each time the trusted device is opened. The specific settings can be made as needed, and this application does not limit this.

[0348] Next, the second processing stage (i.e. organizing materials) is introduced in detail.

[0349] When the global collection application is in full screen mode, the materials collected in the global collection can be displayed. At this time, multiple individual icons and / or multiple group icons may be included in the displayed main interface. Among them, the individual icon is used to indicate a material, and the individual icon can be used to trigger the first device to display the specific content of the material corresponding to the individual icon. The group icon is used to indicate multiple materials grouped as a group, and the group icon can be used to trigger the first device to display the secondary interface corresponding to the group icon, which may include multiple individual icons included in the group, and each individual icon can be used to trigger the first device to display the specific content of the material corresponding to the individual icon.

[0350] In addition, in the main interface and the secondary interface, in addition to displaying individual icons and group icons, the names corresponding to the material or the group of materials will also be displayed below the individual icons and group icons respectively.

[0351] Optionally, since the individual icon and the group icon indicate different contents, in order to visually distinguish them, the individual icon and the group icon can have different display effects. For example, the individual icon can be represented as a rounded square pattern, and the group icon can be represented as multiple overlapping rounded square patterns, which are staggered at a certain angle. Alternatively, the group icon can also be in the form of stacked cards. In addition, a digital number can be displayed in the upper right corner of the group icon, and the size of the number is used to indicate the number of individual materials included in the group material. This is only an example and is not limited in this application.

[0352] Optionally, since the collected materials include multiple types, in order to visually distinguish them intuitively, the individual icons corresponding to different types of materials can have different display effects. For example, when the material type is a document, the corresponding individual icon can use the first character of the document format as the pattern in the icon; when the material type is a video, the corresponding individual icon can be superimposed with a video icon, etc.

[0353] It is understandable that the collected materials are materials of various types. Users may need to use multiple materials of the same type or different types when using them. In order to facilitate the use of users, the collected materials in the global collection can be sorted according to the needs of users, such as grouping multiple materials of the same type into a group, or grouping multiple materials of different types into a group. Alternatively, materials from a certain device source can also be grouped into a group, which is not limited in this application.

[0354] 1. Grouping of materials (e.g. Figure 8A-8F (shown)

[0355] The following example illustrates the process of grouping different types of materials by dragging, combined with the floating state of the global collection.

[0356] Take the mobile phone as the first device as an example. Fig. 8A The display interface 801 shown in the figure takes the public account article as an example. In the partial display area of ​​the display interface, it also includes the floating state of global collection, which is located on the inner side of the right edge of the display interface. The floating state includes multiple individual icons, and the corresponding name is displayed below each individual icon, and a corner mark indicating the device source is displayed in the upper right corner. Fig. 8A As shown, the floating state of the global collection application includes an individual icon corresponding to a material of the video type, namely the first individual icon tb1, "Video Excerpt 02" is the material name corresponding to the individual icon, and a badge indicating that the device source is a laptop is displayed in the upper right corner.

[0357] The mobile phone can detect the user operation on the first body icon tb1, and in response to the user operation, the mobile phone can display the following Figure 8BThe display interface 802 shown, in which the global favorite application floating state can be switched to the sorting window, and the first body icon tb1 can be in the icon selected state. At this time, in the sorting window, the first body icon in the icon selected state can be split and displayed as two individual icons (foreground icon and background icon). The foreground icon is the same as the original first body icon tb1, and the foreground icon can be moved in the display interface 802 with the user's drag operation; the background icon is the same as the original first body icon tb1 except for the color. The background icon does not move with the user's drag operation, or the background icon remains at the original display position of the first body icon tb1.

[0358] Alternatively, if Figure 8B As shown in FIG. 1 , when the first individual icon tb1 is split into two individual icons, a circular selection box including blank content can also be displayed at the bottom right of the other icons, and the circular selection box including blank content is used to indicate that the icon is not selected. A circular selection box including a check mark can also be displayed at the bottom right of the background icon split from the first individual icon tb1, and the circular selection box including a check mark is used to indicate that the current individual icon is selected by the user. In addition, when the first individual icon tb1 is selected by the user, as shown in FIG. Figure 8B As shown, a close control can also be displayed in the upper left corner of the sorting window, and the close control is used to trigger the sorting function of exiting the suspended state.

[0359] Afterwards, if Figure 8C In the display interface 803 shown, in response to the user's dragging operation on the foreground icon in the sorting window, the foreground icon can be moved to the top of another individual icon, such as the second individual icon tb2. The second individual icon tb2 is an individual icon corresponding to a material of the picture type. "fff.jpg" is the material name corresponding to the individual icon. The absence of a superscript in the upper right corner indicates that the device originates from the mobile phone. When the mobile phone detects that the foreground icon corresponding to the first individual icon tb1 overlaps with the projection of the second individual icon tb2, the mobile phone can automatically combine the first individual icon tb1 and the second individual icon tb2 together and switch to the group icon tb3 for display. Fig.8D The display interface 804 is shown, wherein in the sorting window, the group icon tb3 is the group icon corresponding to the first individual icon tb1 and the second individual icon tb2 being grouped together, and "Video Excerpt 02" is the name automatically given to the group icon tb3 by the mobile phone.

[0360] Alternatively, if Figure 8CAs shown, when the electronic device detects that the foreground icon corresponding to the first body icon tb1 overlaps with the projection of the second body icon tb2, a layer of gray blocks can be added to the background of the second body icon tb2 to prompt the user that the second body icon can be grouped together with the first body icon tb1.

[0361] Optionally, as a possible implementation, the group icon can be directly formed by overlapping the included individual icons. As another possible implementation, the group icon can also be formed by shrinking the included individual icons and splicing them together, or in the form of stacked cards. For example, Fig.8D As shown, when the first individual icon tb1 and the second individual icon tb2 form a group, the corresponding group icon tb3 is formed by overlapping the first individual icon tb1 and the second individual icon tb2; or, the group icon tb is formed by splicing the first individual icon tb1 and the second individual icon tb1. The above are only two examples, and the style and color of the group icon can be set as needed, as long as it is visually easy to distinguish from the individual icons.

[0362] Afterwards, the phone can detect the user's operation on the close control in the upper left corner of the sorting window, such as Fig. 8E As shown, in response to the user operation, the electronic device can exit the sorting function and switch from the sorting window to the floating state of the global collection, such as Figure 8F Display interface 805 shown. Compared with the global favorites suspension state in display interface 801, display interface 805 has two individual icons below that are grouped together. Therefore, the two individual icons that are displayed separately in display interface 801 are displayed as a group icon in display interface 805. Apart from this, the other display contents in the two display interfaces are the same.

[0363] It can be understood that the above is an example of grouping materials by dragging when the global collection is in a suspended state.

[0364] Optionally, based on Figure 8F In the display interface 805 shown, the mobile phone can also detect a click operation on the group icon tb3. In response to the click operation, the mobile phone can display the secondary interface corresponding to the group icon tb3. Here, the group icon tb3 is composed of a first individual icon tb1 and a second individual icon tb2. The first individual icon tb1 and the second individual icon tb2 can be displayed in the secondary interface.

[0365] It can be understood that when the global collection is in a suspended state, the process of grouping materials by dragging for an individual material and a group of materials, or for two groups of materials, is similar to the above and will not be repeated here.

[0366] It can be understood that when the global collection is in full screen mode, the process of grouping materials by dragging is similar to the above and will not be repeated here.

[0367] In addition, no matter the global collection is in suspended state or full screen state, it is also possible to trigger the display of a card including a "New Group" option, and form a group by triggering the "New Group" option.

[0368] In the embodiment of the present application, regardless of whether the synchronized materials of the trusted device are downloaded to the local end, they can be grouped.

[0369] It should also be noted that after grouping, the grouping results of the materials included in the above-mentioned floating state will be synchronized to the global collection of the trusted device, that is, the material grouping displayed on other trusted devices is the same as that of the first device.

[0370] 2. Material movement (such as Figure 9A-9E (shown)

[0371] The following example illustrates the material movement process in combination with the suspended state of the global collection.

[0372] Taking the mobile phone as the first device as an example, Fig.9A The display interface 901 shown is Fig. 8A The display interface 801 shown is the same as that shown in FIG. Fig. 9B The display interface 902 shown is Figure 8B The display interface 802 shown is the same. When the global collection is in a suspended state, the process of selecting the first individual icon can refer to the above description of the display interface 801 and the display interface 802, which will not be repeated here.

[0373] After the mobile phone detects that the first body icon tb1 is in the icon selected state, see Fig. 9C In the display interface 903 shown in FIG. 1 , in response to a user dragging operation on a foreground icon split from the first body icon tb1, the foreground icon can be moved to a position overlapping with the third body icon tb3 or the fourth body icon tb4. When the electronic device detects that the foreground icon split from the first body icon tb1 overlaps with the lower projection of the third body icon tb3 and / or the upper projection of the fourth body icon tb4, the electronic device detects that the foreground icon split from the first body icon tb1 overlaps with the lower projection of the third body icon tb3 and / or the upper projection of the fourth body icon tb4. Fig.9D In the display interface 904 shown, the mobile phone can move the fourth individual icon tb4 and all icons below the fourth individual icon tb4 downward to the position of the next adjacent icon in order. Here, the background icon split from the first individual icon tb1 moves together with the other icons.

[0374] Then, the phone can detect that the user lifts his finger and releases the foreground icon. Fig.9EIn the display interface 905 shown, the background icon split from the first body icon tb1 disappears and is no longer displayed, and the first body icon tb is located at the icon position between the third body icon tb3 and the fourth body icon tb4. In addition, if there are other icons below the background icon before it disappears, the other icons will move upward in order to the icon position of the previous icon when the background icon disappears.

[0375] Optionally, combined Fig. 9C and Fig.9D As shown, when the fourth individual icon tb4 and all the icons below move to the adjacent next icon position together, an animation effect of being squeezed away by the first individual icon tb1 may be presented.

[0376] Based on the above drag operation, Fig.9E The display interface 905 shown is relative to Fig.9A For the display interface 901 shown, the position of the first body icon tb1 has moved.

[0377] It should be noted that when the mobile phone detects that the foreground icon split from the first body icon tb1 is dragged upward to a certain position by the user and the user does not release the foreground icon, it can be considered that the icon position displayed in the current sorting window is not the icon position expected by the user after the move. Fig. 9C As shown, the icons currently displayed in the sorting window can automatically scroll down and gradually disappear, while the icons above that were not previously displayed can automatically scroll down and gradually display all of them; until the mobile phone completely displays the first icon or the mobile phone detects that the user's finger is lifted and the foreground icon is released, the scrolling stops. After stopping, the process of the released foreground icon squeezing out other icons is similar to the above, and will not be described here.

[0378] It can be understood that the above is an example of moving materials by dragging when the global collection is suspended.

[0379] It can be understood that when the global collection is in full screen mode, the process of moving materials by dragging is similar to the above and will not be repeated here.

[0380] In addition, no matter the global collection is in suspended state or full screen state, you can also trigger the mobile control to move the material.

[0381] In the embodiment of the present application, the material of the synchronized trusted device can be moved regardless of whether it is downloaded to the local end.

[0382] It should also be noted that after the movement, the material movement results included in the above-mentioned floating state will be synchronized to the global collection of the trusted device, that is, the material movement displayed on other trusted devices is the same as that of the first device.

[0383] 3. Rename the material (such as Figures 10A-10H (shown)

[0384] The following example illustrates the material renaming process in full screen mode of global collections.

[0385] Take the mobile phone as the first device as an example, see Fig. 10A The display interface 1001 shown is the main interface of the global collection. The display interface 1001 includes multiple individual icons, each of which has its corresponding name displayed below it and a badge indicating the source of the device displayed in the upper right corner. Fig. 10A As shown, the display interface 1001 includes a first body icon tb1, and the superscript included in the icon indicates that the material is synchronized with the notebook computer.

[0386] like Fig. 10B As shown, the mobile phone can detect a long press operation on the first body icon tb1; in response to the long press operation, as shown in Fig. 10C In the display interface 1002 shown, the first body icon tb1 may be in an icon selected state (for the convenience of description, it may be referred to as the first body icon tb1 ′). The display interface 1002 may also be referred to as a sorting interface.

[0387] Optionally, when the icon is in the selected state or the non-selected state, the icon can be switched to different colors or the mask can be increased or decreased to distinguish. For example, when the icon is in the selected state, the icon can be switched to gray or the mask can be increased, and when it is in the non-selected state, the icon can be switched to white or the mask can be removed. Fig. 10B and Fig. 10C As shown, the first body icon tb1' in the icon selected state is switched to gray relative to the first body icon tb1.

[0388] Optionally, a circular selection box including a check mark may be displayed at the lower right corner of the first individual icon tb1' in the selected state, and the circular selection box including a check mark is also used to indicate that the current individual icon is in the selected state. Other icons may also display circular selection boxes including blank content at the lower right corners of their respective icons, and the circular selection boxes including blank content are used to indicate that the icon is not in the selected state. Other than that, everything else remains unchanged.

[0389] Optionally, after the mobile phone detects that the first body icon tb1 is in the icon selected state, as a possible implementation method, see Fig. 10CThe display interface 1002 shown in the figure can also display a card kp1 including multiple options, and the multiple options can include, for example, "New Group" and "Rename". Since only one individual icon is in the icon selected state at this time, and an individual icon cannot be grouped, the "New Group" option on the card kp1 needs to be restricted in use and can be set to gray to indicate to the user that a new group cannot be created for an individual icon in the icon selected state. In addition, since the mobile phone detects that only one icon is in the icon selected state, the mobile phone can receive the user's input operation to rename the icon. At this time, the "Rename" option on the card kp1 does not need to be restricted in use and can be set to white to indicate to the user that an individual icon can be renamed. As another possible implementation method, the mobile phone can display a card that only includes the "Rename" option.

[0390] See also Fig. 10D As shown, the mobile phone can detect a click operation on the "Rename" option on the card kp1; in response to the click operation, the electronic device can display the following Fig. 10E The display interface 1003 shown. The display interface 1003 may be composed of four parts, for example, it may include the display interface 1002, the mask layer, the soft keyboard and the card kp5. Among them, the mask layer covers the display interface 1002, which is used to indicate to the user that the operation on the display interface 1002 is restricted at this time; the soft keyboard and the card kp5 are covered on the mask layer, and the soft keyboard can be displayed in the lower half of the display interface 1003 to facilitate receiving the user's input operation; the card kp5 can be displayed in the middle of the display interface 1004 to facilitate display to the user.

[0391] Optionally, the card kp5 may include a title "Rename", an input bar, a cursor, a cancel button and an OK button. The input bar may display the content input by the user in response to the user's input operation; the cursor may indicate the position of the input text; the cancel button is used to trigger the exit of the rename function in response to the user's operation; the OK button is used to trigger the renaming of the icon in the selected state with the text input by the user in the input bar in response to the user's operation.

[0392] It can be understood that, since the name corresponding to the original first body icon is “Video Excerpt 02”, when the card kp5 is displayed in the display interface 1003, the original name “Video Excerpt 02” is displayed in the input field of the card kp5 by default.

[0393] Afterwards, if Fig.10F In the display interface 1004 shown, the electronic device can detect the user's input operation and display the user's input content "promotional material" in the input field; Figure 10G As shown, the electronic device can also detect a user operation on the confirmation button; in response to the user operation, as shown in Fig. 10HAs shown, the electronic device can switch to display interface 1005, in which the name corresponding to the first body icon tb1 is renamed "promotional video".

[0394] Based on the above renaming operation, Fig. 10H The display interface 1005 shown is relative to Fig. 10A For the display interface 1001 shown, the name corresponding to the second-to-last icon is changed from “Video Excerpt 02” to “Promotional Video”.

[0395] It can be understood that the above is an example of renaming materials when the global collection is in full screen mode.

[0396] It can be understood that when the global collection is in a suspended state, the process of renaming the material is similar to the above and will not be repeated here.

[0397] When the global collection is in full screen or floating state, the first device may also display a card including a "rename" option on the interface when detecting a long press operation on a group icon. Based on this, the first device may also detect a user operation on "rename" on the card, and in response to the user operation, the group icon may be renamed. For details, please refer to the above-mentioned process of renaming individual icons, which will not be repeated here.

[0398] In the embodiment of the present application, regardless of whether the material of the synchronized trusted device is downloaded to the local end, it can be renamed.

[0399] It should also be noted that after renaming, the material naming results included in the above-mentioned floating state will be synchronized to the global collection of the trusted device, that is, the material naming displayed on other trusted devices is the same as that of the first device.

[0400] 4. Material deletion (such as Figures 11A-11F (shown)

[0401] The following example illustrates the material deletion process using the full-screen state of global collections.

[0402] Take the mobile phone as the first device as an example, see Fig.11AThe display interface 1101 shown is the main interface of the global collection. The display interface 1101 includes multiple individual icons, each of which has its corresponding name displayed below it, and a superscript indicating the source of the device displayed in the upper right corner. The display interface 1101 includes a first individual icon tb1' in an icon selected state (the superscript included in the icon indicates that the material is synchronized with the laptop computer) and multiple individual icons and multiple group icons in a non-icon selected state, wherein the individual icon in a non-icon selected state may include a second individual icon tb2 (the icon does not include a superscript indicating that the material originates from the mobile phone itself). In addition, a circular selection box including a check mark may be displayed in the lower right corner of the first individual icon tb1' in an icon selected state, and circular selection boxes including blank content may be displayed in the lower right corners of other icons.

[0403] like Fig.11A As shown, the first device can detect a click operation on the second individual icon tb2; Fig. 11B In the display interface 1102 shown, in response to the click operation, the second individual icon tb2 may be in an icon selected state (for ease of description, it may be referred to as the second individual icon tb2'). The second individual icon tb2' in the icon selected state may display a circular selection box including a check mark.

[0404] At this time, the bottom of the display interface 1102 may also include a bottom status bar, which may include: share, select all, move, delete and more controls, among which the share control is used to trigger the material corresponding to the icon that is in the selected state to be exported or shared to other applications in a certain format, the select all control is used to trigger all icons to be in the selected state; the move control is used to trigger the movement of the material, the delete control is used to trigger the deletion of the material, and more controls can trigger more other functions, which are not limited in this application.

[0405] After the first device detects that the first body icon tb1 and the second body icon tb2 are in the icon selected state, Fig. 11C As shown, the first device may detect a user operation for deleting a control; in response to the user operation, the first device may display Fig.11D The display interface 1103 shown. The display interface 1103 may be composed of four parts, for example, including the display interface 1102, the mask layer, the soft keyboard and the card kp6. Among them, the mask layer covers the display interface 1102, which is used to indicate to the user that the operation on the display interface 1102 is restricted at this time; the soft keyboard and the card kp6 are covered on the mask layer, and the soft keyboard can be displayed in the lower half of the display interface 1103 to facilitate receiving the user's input operation; the card kp6 can be displayed in the middle of the display interface 1103 to facilitate display to the user.

[0406] Optionally, the card kp6 may include a prompt "Do you want to delete the two collections?", a cancel button and an OK button. The number in the prompt may be determined based on the number of icons in the selected state; the cancel button is used to trigger the exit deletion function; and the OK button is used to trigger the deletion of the material corresponding to the icon in the selected state.

[0407] It is understandable that in order to avoid user misoperation, after detecting the operation on the delete control, it is necessary to prompt the user once and determine whether the user's intention is true a second time.

[0408] Afterwards, if Fig.11E As shown, the first device can detect a user operation on the confirmation button, and in response to the user operation, see Fig.11F In the display interface 1104 shown, the first body icon tb1 and the second body icon tb2 disappear, and the corresponding materials are deleted.

[0409] In addition, optionally, in the display interface 1104, the first device may also display a card kp6 including a prompt, for example, "2 materials have been deleted" to prompt the user that the deletion operation has been completed.

[0410] Based on the above deletion operation, Fig.11F The display interface 1104 shown is relative to Fig.11A For the display interface 1101 shown, the last icon and the second last icon are reduced.

[0411] It can be understood that the above is an example of deleting materials when the global collection is in full screen mode.

[0412] It can be understood that when the global collection is in a suspended state, the process of deleting the material is similar to the above and will not be repeated here.

[0413] When the global collection is in full screen or floating state, the first device detects a deletion operation on a group icon in a selected icon state, and can also display a card on the interface to prompt the user whether to delete the group of materials; in response to a click operation on the confirmation control on the card, the first device can delete a group of materials corresponding to the group icon. For details, please refer to the above-mentioned process of deleting materials, which will not be repeated here.

[0414] In the embodiment of the present application, the material of the synchronized trusted device can be deleted regardless of whether it is downloaded to the local end.

[0415] It should be noted that after deletion, the remaining material included in the above floating state will be synchronized to the global collection of the trusted device, that is, the same material is also deleted on other trusted devices, and the remaining material displayed is the same as that of the first device.

[0416] Optionally, since the deletion operation will delete the same material on all devices in the same trust ring, when performing the deletion operation, the user may be prompted "Do you want to delete the two collections? After deletion, all devices in the same trust ring will be deleted." After deletion, the user may also be prompted "The material has been deleted on all devices in the same trust ring." The specific text may be changed, and this application does not limit this.

[0417] Finally, the third processing stage (material use and material sharing) is introduced in detail. Figures 12A-12F As shown, Figures 13A-13E shown).

[0418] 1. Use of Materials (such as Figures 12A-12F (shown)

[0419] Take the mobile phone as the first device as an example. Fig. 12A The display interface 1201 shown in the figure may include a note content editing interface in the note application and a floating state of global collections. The floating state may include multiple icons (including individual icons and group icons), and each icon also has its corresponding name displayed below it, and a badge indicating the device source is displayed in the upper right corner. Fig. 12A The floating state shown in the figure includes a group icon tb8, a corresponding name "building.jpg" is displayed below the group icon, and a badge indicating that the device source is a desktop computer is displayed in the upper right corner.

[0420] The mobile phone can detect the user operation on the group icon tb8, and in response to the user operation, the mobile phone can display the following Fig. 12B The display interface 1202 shown in the figure, in which the floating state of the global favorite application can be switched to the sorting window, and the group icon tb8 can be in the icon selected state. At this time, in the sorting window, the group icon tb8 in the icon selected state can be split and displayed as two group icons (foreground icon and background icon). For the specific introduction of the foreground icon and the background icon, and the group icons in the icon selected state and the non-icon selected state, please refer to the above content, which will not be repeated here. Fig. 12B As shown, a close control can also be displayed in the upper left corner of the sorting window, and the close control is used to trigger the sorting function of exiting the suspended state.

[0421] Afterwards, the electronic device may further detect a user operation on the group icon tb8. In response to the user operation, when the electronic device detects that the group icon has moved to the edge of the global collection floating state, the following may be displayed: Fig. 12CThe display interface 1203 shown. Compared with the display interface 1202, the foreground icon corresponding to the group icon tb8 in the display interface 1203 follows the user's operation and moves to the edge of the global collection suspension state. At this time, the background icon corresponding to the group icon tb8 disappears, and the positions of other icons in the suspension state are automatically sorted; in addition, a mark "+" can be displayed in the upper left corner of the foreground icon corresponding to the group icon tb8 to prompt the user that the material corresponding to the group icon tb8 can be added to the note content included in the display interface 1203. It can be understood that the addition mark is only used for prompting and cannot be triggered.

[0422] Then, when the electronic device detects that the foreground icon corresponding to the group icon tb8 moves out of the global collection suspension state, it can display the following Fig.12D The display interface 1204 shown. Compared with the display interface 1203, the foreground icon corresponding to the group icon tb8 follows the user operation, moves out of the global collection suspension state and moves to a certain position of the note content included in the display interface. At this time, the global collection suspension state is hidden and not displayed.

[0423] Optionally, during the process of switching from display interface 1203 to display interface 1204 , the global favorite floating state may disappear with an animation effect of retracting from left to right.

[0424] Optionally, when the image has been downloaded to the mobile phone, the mobile phone detects that the user lifts his finger and releases the foreground icon corresponding to the group icon tb8, such as Fig.12F In the display interface 1206 shown in the figure, the two images indicated by the group icon tb8 can be added to the release position in the note content in the order of icons, and displayed in a size suitable for the note content. When the image is not downloaded to the mobile phone, the mobile phone can download it first and then import it when it detects that the user's finger is lifted and the foreground icon corresponding to the group icon tb8 is released. Fig.12E The display interface 1205 shown in the figure has the group icon tb8 hidden compared to the display interface 1204, and a display mask is superimposed on the note content, and a display card kp7 is superimposed on the mask; the card kp7 can display the text "Material downloading and importing" and a download progress bar. When the download is completed, the display interface 1205 is switched to the display interface 1206.

[0425] Optionally, after the material is imported, the floating state of the global collection can be redisplayed in the display interface 1206. It can be understood that since the material corresponding to the group icon tb8 has been added to the note content, the floating state in the display interface 1206 no longer includes the group icon tb8 compared to the floating state in the display interface 1201, and the other icon contents remain the same.

[0426] It can be understood that the above is an example of using cross-end synchronized materials when the global collection is in a suspended state.

[0427] 2. Material sharing (such as Figures 13A-13E (shown)

[0428] Fig.13A The display interface 1301 shown is Fig. 11B The display interface 1102 shown is similar and will not be described in detail here. When the global collection is in full screen mode, the process of selecting individual icons tb3 and tb4 can refer to the above process for selecting individual icons tb4. Figure 11A-11B The description of the individual icon tb3 is omitted here. The individual icon tb3 includes a superscript indicating that the material is synchronized with the laptop computer, and the individual icon tb4 does not include a superscript indicating that the material is from the mobile phone.

[0429] After the electronic device detects that the individual icon tb3 and the individual icon tb4 are in the icon selected state (for the convenience of description, they can be referred to as individual icon tb3' and individual icon tb4'), as shown in FIG. Fig. 13B As shown, the electronic device can detect the user operation on the sharing control; in response to the user operation, since the materials indicated by the individual icon tb1' and the individual icon tb2' are both picture types, when the electronic device shares multiple materials of the picture type, it can switch to display as shown in FIG. Fig. 13B The display interface 1302 shown. The display interface 1302 may be composed of three parts, for example, it may include the original display interface 1302, the mask layer and kp8. The mask layer covers the display interface 1302 to indicate to the user that the operation on the display interface 1302 is restricted at this time; the card kp8 covers the mask layer and is displayed in the lower half of the display interface 1303 to facilitate receiving the user's input operation.

[0430] Optionally, the card kp8 may include but is not limited to two options: "Merge into a long picture to share" and "Share separately", which provide users with choices based on the number of shares. Among them, "Merge into a long picture to share" can splice the picture materials corresponding to multiple icons in the icon selected state, and switch them to a long picture for sharing. "Share separately" can share the picture materials corresponding to multiple icons in the icon selected state without splicing them.

[0431] like Fig. 13C As shown, the electronic device may detect a click operation on the option of "merge into a long picture to share"; optionally, as a possible implementation, in response to the click operation, the electronic device may display the following Fig.13EDisplay interface 1304 is shown. Compared with display interface 1302, only card kp8 is switched to card kp10 in display interface 1304, and other contents remain unchanged.

[0432] As another possible implementation, in response to a click operation on the option "Merge into long picture to share", the electronic device may first display the following Fig.13D The display interface 1303 shown in FIG. Fig.13E The display interface 1304 is shown. Compared with the display interface 1302, the display interface 1303 only switches the card kp8 to the card kp9, and the other contents remain unchanged. The card kp9 may include the text "loading" to prompt the user that the long image is being merged. It is understandable that since the image material is usually large in data, it takes a little longer to merge into a long image. Therefore, in order to avoid the user mistakenly thinking that the electronic device has not processed it, this feedback can be added to prompt.

[0433] After the electronic device successfully merges the long image, it can be displayed as follows Fig.13E The display interface 1304 shown. In the card kp10 included in the display interface 1304, four display areas may be included from top to bottom, such as a first display area 131, a second display area 132, a third display area 133 and a fourth display area 134.

[0434] Optionally, the first display area 131 may display an icon corresponding to the merged long image, the word "long image", the name of the long image, the data size of the long image, and the format of the long image; a small triangle may also be displayed next to the format of the long image, which may be called a drop-down menu control, for displaying a drop-down menu and providing more image format options for the user to select. In other words, the image format of the long image may be switched by the user's selection.

[0435] Optionally, the second display area 132 may include device icons of multiple devices, which have established a trusted device connection with the present device, and the present application does not limit the connection method. The multiple device icons may be used to trigger sending the merged long image to the corresponding device. For example, the multiple device icons may include "X30 of user 1", and subsequently, in response to a click operation on the device icon, the electronic device may send the merged long image to the device indicated by "X30 of user 1".

[0436] Optionally, the third display area 133 may include multiple application icons, which may be used to trigger sharing of the merged long image to corresponding applications. For example, the multiple application icons may include a browser, contacts, mail, notes, and gallery, etc. Subsequently, in response to a click operation on the note application icon, the electronic device may send the merged long image to the note application.

[0437] Optionally, the fourth display area 134 may include two options, "Save" and "Print". "Save" can store the merged long image on the device without sharing it. "Print" can send the merged long image to a connected printer for printing.

[0438] It can be understood that the above is an example of sharing materials of the same type but with different source information on two devices in full screen mode.

[0439] It can be understood that the process of selecting multiple devices with the same or different source information and multiple materials of the same or different types for sharing is similar to the above and will not be repeated here.

[0440] It can be understood that when the global collection is in a suspended state, the process of sharing multiple materials is similar to the above. Please refer to the above description and will not be repeated here.

[0441] It should be supplemented that if two or more document types are selected for sharing, and the document format is PDF, then in response to a click operation on the sharing control, the electronic device can display a card kp8, and the card kp8 at this time may include but is not limited to the two options of "Merge as PDF to share" and "Share separately", and the two options provide users with choices from the dimension of the number of shares. Correspondingly, the content of the first display area in the card kp10 can display the icon corresponding to the merged PDF, the text "Merge", the name of the merged PDF (the name of the first document to be merged can be selected by default), the data size of the merged PDF, and the format of the merged document, such as PDF; a small triangle can also be displayed next to the document format, which can be called a drop-down menu control, which is used to display a drop-down menu and provide more document format options for the user to choose. In other words, the merged document format can be switched by the user's selection.

[0442] It should also be noted that if only one material (regardless of the type) is selected, the electronic device may not display the card kp8 and directly select the material from the Fig.13A The display interface 1301 shown switches to Fig.13E The display interface 1302 shown in the figure may not include the content of the first display area 131 .

[0443] Optionally, Fig.13AThe bottom status bar in the display interface 1301 shown may also include a merge control; when the icon corresponding to a material is in the selected state, the merge control may be grayed out to restrict its use; when the icons corresponding to multiple materials are in the selected state, the merge control may be restored to use, thereby, in response to a click operation on the merge control, multiple materials in the selected state may be merged, and the merged material may be automatically collected into the global collection by default.

[0444] The sharing method provided in this application combines multiple sharing needs such as quantity, format, device, application, saving and printing, and provides users with choices from the quantity dimension, format dimension, device dimension and application dimension to meet various user needs and improve user experience.

[0445] Fig.14 An exemplary flow chart of a material processing method provided in an embodiment of the present application.

[0446] In some possible cases, the exemplary process involved in the electronic device processing the material can refer to the following description of steps S1401 to S1407.

[0447] S1401: The electronic device displays a first interface, where the first interface includes a first material.

[0448] It is understandable that the first interface can be the interface involving the aforementioned content displaying text content (such as the main interface of a news application), or the first interface can also be the interface involving the aforementioned content displaying pictures (such as the main interface of a news application).

[0449] In a possible implementation, the first interface may be Figure 3A In the display interface 301 shown, the first material is a text a1. At this time, the first material is a text type.

[0450] S1402: In response to the first stage of the first operation on the first material, start a target application, and display a hidden icon corresponding to the target application in a floating manner on the first interface.

[0451] It can be understood that the target application is the global favorite application mentioned above.

[0452] It can be understood that the first interface displaying the hidden icon corresponding to the target application can be an interface in which the text involved in the aforementioned content is in a selected state and the hidden icon of the global collection appears, or the first interface displaying the hidden icon corresponding to the target application can be an interface in which the picture involved in the aforementioned content is in a selected state and the hidden icon of the global collection appears.

[0453] In a possible implementation, the first stage of the first operation may be Figure 3BThe second interface can be Figure 3B Display interface 302 is shown.

[0454] S1403: In response to the second stage of the first operation on the first material, the hidden state icon is switched to a collection state icon corresponding to the target application, and the first material is collected into the target application through the collection state icon.

[0455] In a possible implementation, the second stage of the first operation may be Figure 3C , Figure 3D The dragging or throwing motion shown.

[0456] S1404: After the first material is collected, the collection state icon is switched to a hidden state icon.

[0457] S1405. In response to a second operation on the collected hidden state icon, the hidden state icon is switched to a floating state icon corresponding to the target application, and the floating state icon is displayed in a floating state on the first interface. The floating state icon includes a first icon indicating a first material.

[0458] In a possible implementation, the second operation may be Figure 3G The click operation shown, the first icon can be Figure 3H Icon 3071 shown.

[0459] S1406: Before the first material is collected, the second electronic device displays a second interface, and a hidden icon corresponding to the target application is displayed in a floating manner on the second interface.

[0460] It is understandable that at this time, the global favorites application has been started and the hidden icon is always displayed.

[0461] In a possible implementation, the second interface may be Fig. 3I Display interface 308 is shown.

[0462] S1407: After the first material is collected, the second electronic device detects a third operation on the hidden icon in the second interface.

[0463] In a possible implementation, the third operation may be Fig. 3I Click actions shown.

[0464] S1408. In the second electronic device, in response to the third operation, the hidden state icon is switched to a floating state icon, and the floating state icon is displayed in a floating state on the second interface. The floating state icon includes a first icon indicating a first material and a first superscript.

[0465] In a possible implementation, the first icon and the first superscript may be the icon 3091 and the superscript at the upper right corner of the icon 3091 shown in FIG. 3J.

[0466] The relevant contents involved in step S1401 to step S1408 can refer to the description of the relevant contents of collecting text in the collection stage and synchronously displaying it in the floating icon of the tablet, which will not be repeated here. The exemplary interface involved in step S1401 and step S1408 can refer to the description of the relevant contents of the collection stage in the previous text. Figure 3A-3J ,as well as Figure 3A-3J The description of the relevant contents will not be repeated here.

[0467] The following is an introduction to the device involved in the embodiments of the present application.

[0468] Fig.15 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0469] The electronic device may include a processor 110 , an external memory interface 120 , an internal memory 121 , an audio module 130 , a speaker 130A, a receiver 130B, a microphone 130C, an earphone interface 130D, a display screen 140 , a camera 150 , and a touch sensor 160 .

[0470] The structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the modules illustrated in the embodiments of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device.

[0471] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. A memory may also be provided in the processor 110 for storing instructions and data.

[0472] The electronic device implements the display function through the GPU, the display screen 140, and the application processor.

[0473] The GPU is a microprocessor for image processing, which is connected to the display screen 140 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information. The display screen 140 is used to display images, videos, etc. In some embodiments, the electronic device may include 1 or N display screens 140, where N is a positive integer greater than 1.

[0474] In the embodiment of the present application, the electronic device displays the aforementioned Figure 3A-3J , Figure 4A-Figure 4I , Figure 5A-5H , Figure 6A-6D , Figure 7A-Figure 7I , Figure 8A-8F , Figure 9A-9E , Figures 10A-10H , Figures 11A-11F , Figures 12A-12F , Figures 13A-13E The capabilities of the user interface shown rely on the display functions provided by the GPU, display screen 140, and application processor described above.

[0475] The camera 150 is used to capture still images or videos. The ISP is used to process the data fed back by the camera 150. Light is transmitted to the camera photosensitive element through the lens, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The electronic device may include 1 or N cameras 150, where N is a positive integer greater than 1. The video codec is used to compress or decompress digital video. The electronic device may support one or more video codecs. In this way, the electronic device can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0476] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data. The non-volatile memory can also store executable programs and store user and application data, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.

[0477] In the embodiment of the present application, the code for implementing the information processing method described in the embodiment of the present application may be stored in a non-volatile memory. When running the camera application, the electronic device may load the executable code stored in the non-volatile memory into the random access memory.

[0478] The external memory interface 120 may be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device.

[0479] The electronic device can implement audio functions through the audio module 130, the speaker 130A, the receiver 130B, the microphone 130C, the headphone interface 130D, and the application processor.

[0480] The audio module 130 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The speaker 130A, also called "speaker", is used to convert audio electrical signals into sound signals. The receiver 130B, also called "earpiece", is used to convert audio electrical signals into sound signals. The microphone 130C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. The headphone interface 130D is used to connect wired headphones.

[0481] In the embodiment of the present application, when the speech-to-text function is turned on, the electronic device can enable the microphone 130C to collect sound signals and convert the sound signals into corresponding text.

[0482] The touch sensor 160 is also called a "touch control device". The touch sensor 160 can be arranged on the display screen 140. The touch sensor 160 and the display screen 140 form a touch screen, also called a "touch control screen". The touch sensor 160 is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 140. In other embodiments, the touch sensor 160 can also be arranged on the surface of the electronic device, which is different from the position of the display screen 140.

[0483] In the embodiment of the present application, the electronic device can use the touch sensor 160 to detect the user's click, slide and other operations on the display screen 140 to achieve Figure 3A-3J , Figure 4A-Figure 4I , Figure 5A-5H , Figure 6A-6D , Figure 7A-Figure 7I , Figure 8A-8F , Figure 9A-9E , Figures 10A-10H , Figures 11A-11F , Figures 12A-12F , Figures 13A-13E The cross-end interaction method shown.

[0484] Fig.16 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0485] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device.

[0486] It should be understood that the layered architecture can divide the software into several layers, each layer has a clear role and division of labor; the layers can communicate with each other through software interfaces.

[0487] like Fig.16 As shown, the Android system can be divided into four layers: from top to bottom, they are application layer, application framework layer, Android runtime and system library, and kernel layer. The application layer can include a series of application packages.

[0488] like Fig.16 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0489] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0490] like Fig.16 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0491] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0492] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0493] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0494] The phone manager is used to provide communication functions for electronic devices, such as the management of call status (including answering, hanging up, etc.).

[0495] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0496] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0497] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0498] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0499] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0500] The system library may include multiple functional modules, such as: surface manager, media library, 3D graphics processing library (such as open graphics library for embedded systems (OpenGL ES)) and 2D graphics engine (such as skia graphics library (SGL)).

[0501] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0502] The media library supports playback and recording of multiple audio formats, playback and recording of multiple video formats, and still image files. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, moving picture experts group audio layer III (MP3), advanced audio coding (AAC), adaptive multi-rate (AMR), joint photographic experts group (JPG) and portable network graphics (PNG).

[0503] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0504] A 2D graphics engine is a drawing engine for 2D drawings.

[0505] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0506] In the embodiment of the present application, the electronic device can use the above software system to realize the display of the above Figure 3A-3J , Figure 4A-Figure 4I , Figure 5A-5H , Figure 6A-6D , Figure 7A-Figure 7I , Figure 8A-8F , Figure 9A-9E , Figures 10A-10H , Figures 11A-11F , Figures 12A-12F , Figures 13A-13E The cross-end interaction method shown.

[0507] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A cross-terminal interaction method, characterized in that: Applied to a cross-end interaction system including a first electronic device and a second electronic device, the method includes: The first electronic device displays a first interface, where the first interface includes a first material; In the first electronic device, in response to the first stage of the first operation on the first material, a target application is started, and a hidden icon corresponding to the target application is suspended and displayed on the first interface; In the first electronic device, in response to the second stage of the first operation on the first material, the hidden state icon is switched to a collection state icon corresponding to the target application, and the first material is collected into the target application through the collection state icon; the size of the hidden state icon is smaller than the size of the collection state icon, the collection state icon is displayed in a floating manner on the first interface, and the first stage and the second stage are continuous; In the first electronic device, after the first material is collected, the collection state icon is switched to the hidden state icon; In the first electronic device, in response to a second operation on the collected hidden state icon, the hidden state icon is switched to a floating state icon corresponding to the target application, the floating state icon is displayed in a floating state on the first interface, and the floating state icon includes a first icon indicating the first material; Before the first material is collected, the second electronic device displays a second interface, wherein the hidden icon corresponding to the target application is suspended and displayed on the second interface; After the first material is collected, the second electronic device detects a third operation on the hidden icon in the second interface; In the second electronic device, in response to the third operation, the hidden state icon is switched to the floating state icon, and the floating state icon is displayed floating on the second interface. The floating state icon includes the first icon and a first superscript indicating the first material, and the first superscript is used to indicate that the first material comes from an electronic device other than the second electronic device.

2. The method according to claim 1, characterized in that: The floating icon in the second electronic device includes a first control, and after the floating icon is displayed in a floating state on the second interface, the method further includes: In the second electronic device, in response to a fourth operation on the first control, the second interface suspends and displays a first card, where the first card at least includes a plurality of callable service options corresponding to the first electronic device, and the callable service options include a shooting option; In the second electronic device, in response to a fifth operation on the shooting option, the second electronic device sends a first instruction to the first electronic device, where the first instruction is used to instruct to start a camera application of the first electronic device; After receiving the first instruction, the first electronic device starts the camera application and displays a shooting interface; In the second electronic device, the first card is switched to a multi-screen collaborative interface, and the multi-screen collaborative interface is displayed floating on the second interface, and the multi-screen collaborative interface includes the shooting interface corresponding to the first electronic device.

3. The method according to claim 2, characterized in that The shooting interface includes a shooting control, and the method further includes: In the second electronic device, in response to a sixth operation on the shooting control included in the multi-screen collaborative interface, a second instruction is sent to the first electronic device, where the second instruction is used to instruct shooting; After receiving the second instruction, the first electronic device shoots and collects the shot second material into the target application of the first electronic device; After the second material is collected, the floating icon of the first electronic device includes a second icon indicating the second material; The first electronic device sends the second icon and a second superscript to the second electronic device, where the second superscript is used to indicate that the second material comes from another electronic device except the second electronic device.

4. The method according to claim 2 or 3, characterized in that: The callable service option also includes a recording option, and the method further includes: In the second electronic device, in response to the seventh operation on the recording option, the second electronic device sends a third instruction to the first electronic device, where the third instruction is used to instruct to start a recording application of the first electronic device; After receiving the third instruction, the first electronic device starts the recording application and displays a recording interface; In the second electronic device, the first card is switched to the multi-screen collaborative interface, and the multi-screen collaborative interface is displayed floating on the second interface. The multi-screen collaborative interface includes the recording interface corresponding to the first electronic device.

5. The method according to claim 4, characterized in that After receiving the third instruction, the first electronic device starts the recording application and displays a recording interface, including: The first electronic device displays a lock screen interface; In the first electronic device, in response to an unlocking operation on the lock screen interface, starting the recording application and displaying the recording interface; In the second electronic device, the multi-screen collaborative interface includes the recording interface corresponding to the first electronic device, including: When the first electronic device displays the lock screen interface, the multi-screen collaboration interface in the second electronic device displays a lock screen prompt interface, where the lock screen prompt interface is used to prompt the first electronic device to be locked; When the first electronic device starts the recording application and displays the recording interface, in the second electronic device, the lock screen prompt interface included in the multi-screen collaborative interface is switched to the recording interface corresponding to the first electronic device.

6. The method according to claim 4 or 5, characterized in that: The recording interface includes a recording control, and the method further includes: In the second electronic device, in response to an eighth operation on the recording control included in the multi-screen collaborative interface, sending a fourth instruction to the first electronic device, where the fourth instruction is used to instruct recording; After receiving the fourth instruction, the first electronic device records the audio and collects the third material obtained by the recording into the target application of the first electronic device; After the third material is collected, the floating icon of the first electronic device includes a third icon indicating the third material; The first electronic device sends the third icon and the third superscript to the second electronic device, where the third superscript is used to indicate that the third material comes from another electronic device except the second electronic device.

7. The method according to any one of claims 2 to 6, characterized in that The callable service information further includes a handwriting option, and the method further includes: In the second electronic device, in response to a ninth operation on the handwriting option, the second electronic device sends a fifth instruction to the first electronic device, where the fifth instruction is used to instruct to start a note application of the first electronic device; After receiving the fifth instruction, the first electronic device starts the note application and displays a handwriting input interface; In the second electronic device, the first card is switched to the multi-screen collaborative interface, and the multi-screen collaborative interface is displayed floating on the second interface. The multi-screen collaborative interface includes the handwriting input interface corresponding to the first electronic device.

8. The method according to claim 7, characterized in that After receiving the fifth instruction, the first electronic device starts the note application and displays the handwriting input interface, including: The first electronic device displays a lock screen interface; In the first electronic device, in response to an unlocking operation on the lock screen interface, the first electronic device starts the note application and displays the handwriting input interface; In the second electronic device, the multi-screen collaborative interface includes the handwriting input interface corresponding to the first electronic device, including: When the first electronic device displays the lock screen interface, the multi-screen collaboration interface in the second electronic device displays a lock screen prompt interface, where the lock screen prompt interface is used to prompt the first electronic device to be locked; When the first electronic device starts the note application and displays the handwriting input interface, in the second electronic device, the lock screen prompt interface included in the multi-screen collaborative interface is switched to the handwriting input interface corresponding to the first electronic device.

9. The method according to claim 7 or 8, characterized in that: The handwriting input interface includes a handwriting input area and a completion control, and the method further includes: In the second electronic device, in response to an input operation on the handwriting input area included in the multi-screen collaborative interface, sending handwriting input content to the first electronic device; In response to a tenth operation on the completion control, sending a sixth instruction to the first electronic device, wherein the sixth instruction is used to indicate that the handwriting input is completed; After receiving the sixth instruction, the first electronic device collects the fourth material obtained according to the received handwriting input content into the target application of the first electronic device; After the fourth material is collected, the floating icon of the first electronic device includes a fourth icon indicating the fourth material; The first electronic device sends the fourth icon and the fourth superscript to the second electronic device, where the fourth superscript is used to indicate that the fourth material comes from another electronic device except the second electronic device.

10. The method according to claim 3, 6 or 9, characterized in that: The multi-screen collaborative interface also includes an exit control, and the method further includes: In the second electronic device, in response to an eleventh operation on the exit control, the multi-screen collaborative interface is closed and switched to the floating state icon, and the floating state icon is displayed in a floating state on the second interface; The floating state icons of the second electronic device include a second icon and a second superscript, or include a third icon and a third superscript; or include a fourth icon and a fourth superscript.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Before the first material is collected, the second electronic device displays a second interface, the second interface includes a sidebar displayed in a suspended manner, and the sidebar includes an application icon corresponding to the target application; After the first material is collected, the second electronic device detects a twelfth operation on the application icon; In the second electronic device, in response to the twelfth operation, the target application is started, the sidebar is switched to the floating icon, the floating icon is displayed in a floating state on the second interface, and the floating icon includes the first icon and the first superscript; When the second electronic device starts the target application for the first time, the floating icon further includes a prompt card, and the prompt card is used to prompt that the first material comes from another electronic device except the second electronic device; In the second electronic device, in response to a thirteenth operation on the top horizontal bar control included in the floating state icon, the floating state icon is switched to the hidden state icon, and the hidden state icon is displayed in a floating state on the second interface.

12. The method according to claim 3 or 10, characterized in that: The method further comprises: The second electronic device displays a third interface, the third interface includes the floating icon displayed in a floating state, the floating icon includes the first icon, the first superscript, the second icon and the second superscript; In the second electronic device, in response to the fourteenth operation, after the first icon moves to the second icon position or after the second icon moves to the first icon position, the first icon and the second icon switch to a first group of icons; In the first electronic device, the first icon and the second icon are synchronously switched to the first group of icons.

13. The method according to claim 3, 10 or 12, characterized in that The method further comprises: The second electronic device displays a fourth interface, the fourth interface includes the floating icon displayed in a floating state, the floating icon includes the first icon, the first superscript, the second icon and the second superscript; In the second electronic device, in response to a fifteenth operation, the first icon or the second icon moves within the floating icon; In the first electronic device, the first icon or the second icon moves synchronously.

14. The method according to any one of claims 3 to 13, characterized in that The method further comprises: The second electronic device displays a full-screen interface corresponding to the target application; the first icon included in the full-screen interface is in a selected state, and the full-screen interface also includes a rename control; In the second electronic device, in response to a sixteenth operation on the rename control, a fifth interface is displayed, wherein the fifth interface includes an input field; In the second electronic device, in response to an input operation on the input field, naming the first icon as a target name; In the second electronic device, after naming, the full-screen interface includes the first icon whose name is the target name; In the first electronic device, the name of the first icon is synchronized with the target name.

15. The method according to claim 3, characterized in that The method further comprises: The second electronic device displays a full-screen interface; the first icon and the second icon in the full-screen interface are in a selected state, and the full-screen interface further includes a delete control; In the second electronic device, in response to a seventeenth operation on the deletion control, deleting the first material and the second material; In the second electronic device, after the deletion, the full-screen interface does not include the first icon and the second icon; In the first electronic device, the first material and the second material are deleted synchronously, and after the deletion, the first icon and the second icon are not included.

16. The method according to any one of claims 3 to 15, characterized in that After the floating icon includes a first icon indicating the first material and the first superscript, the method further includes: In the second electronic device, in response to the interface switching operation on the second electronic device, the second electronic device displays a sixth interface, and the floating icon is displayed in a floating state on the sixth interface; In the second electronic device, in response to the eighteenth operation on the first icon, the second electronic device downloads the first material indicated by the first icon and displays a seventh interface, the seventh interface includes the first material, and the floating icon is displayed floating on the seventh interface, and the floating icon does not include the first icon and the first superscript.

17. The method according to any one of claims 3 to 16, characterized in that The method further comprises: The second electronic device displays a full-screen interface; the first icon and the fifth icon in the full-screen interface are in an icon selected state, and the full-screen interface also includes a sharing control; the fifth icon is used to indicate the fifth material collected by the target application in the second electronic device; In the second electronic device, in response to a nineteenth operation on the sharing control, a second card is displayed, wherein the second card includes a plurality of options indicating different formats, different devices, different applications, saving, and printing; In the second electronic device, in response to a twentieth operation on the second card, the first material and the fifth material are shared based on the selected option.

18. The method according to claim 17, characterized in that After responding to the nineteenth operation on the sharing control and before displaying the third card, the method further includes: displaying a third card, the third card comprising a merge option; In response to a twenty-first operation on the merge option, merge the first material and the fifth material; The first material and the fifth material are shared based on the selected option, including: The merged first material and the fifth material are shared based on the option selected.

19. The method according to any one of claims 1 to 18, characterized in that The method further comprises: In the second electronic device, in response to a twenty-second operation on the first icon, the first material is downloaded and displayed.

20. The method according to any one of claims 1 to 19, characterized in that In the first electronic device, the collection state icon is switched to the hidden state icon, including: The collection state icon is switched to a feedback state icon corresponding to the target application, and the feedback state icon is used to indicate that the collection has been successful; The feedback state icon is switched to the hidden state icon.

21. The method according to any one of claims 1 to 20, characterized in that The types of the first material include at least: text, graphics, images, audio, and tables; the first stage of the first operation is a long press operation, and the second stage of the first operation is dragging or throwing.

22. The method according to any one of claims 1 to 21, characterized in that The hidden state icon is in a semi-transparent semicircle shape, and the collection state icon is in a capsule shape or a window shape.

23. The method according to claim 10, characterized in that The first superscript, the second superscript, the third superscript and the fourth superscript are the same.

24. An electronic device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, wherein the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method executed by the first electronic device or the second electronic device as described in any one of claims 1 to 23.

25. A cross-terminal interactive system, characterized in that: The cross-end interaction system includes a first electronic device and a second electronic device, and the first electronic device and the second electronic device execute the method as described in any one of claims 1 to 23.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 23.

Images