Interface element display method and device
By displaying floating interface elements on the terminal, the cumbersome interaction process when viewing or sharing information across applications is solved, and more efficient human-computer interaction performance is achieved.
Patent Information
- Application Number
- CN202011564390.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-25
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-12-25
AI Technical Summary
In terminals such as mobile phones, when viewing or sharing information across applications, the interaction process is cumbersome, the operation steps are numerous, and the human-computer interaction performance is poor.
By displaying floating interface elements on the terminal's display screen, supporting drag and drop, copying and other operations, the interface elements are displayed in the form of a floating UI, realizing cross-application information viewing and file transfer, and simplifying the interactive process.
It improves the convenience of cross-application information viewing and sharing operations, reduces operation steps, and improves human-computer interaction performance.
Smart Images

Figure CN114756151B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic devices, and in particular to a method and device for displaying interface elements. Background Art
[0002] At present, mobile phones and other terminals have become electronic devices that are frequently used in people's daily lives and work. By installing applications (APPs, hereinafter referred to as applications) in mobile phones, the functions of mobile phones can be improved to provide users with a better user experience. However, in the process of daily use of mobile phones by users, if they need to view or share information across applications, there will be problems such as cumbersome interaction processes and many operation steps, and poor human-computer interaction performance.
[0003] For example, when user 1 is using a shopping app on a mobile phone and wants to share a picture of the product currently being browsed with user 2, he needs to first save the picture of the product to the gallery app on the mobile phone, then open the instant messaging app on the mobile phone to find user 2, open the chat interface with user 2, then open the gallery app through the quick entry on the chat interface, select the saved picture of the product, and finally click send to share the picture of the product with user 2 through the instant messaging app. For another example, user A uses the shopping app on his mobile phone to buy a product, and needs to go through multiple steps to check whether the product has been shipped, the logistics of the product, and other information. Summary of the invention
[0004] The embodiments of the present application provide an interface element display method and device, which solve the problems of complicated interaction process, many operation steps and poor human-computer interaction performance when viewing or sharing information across applications.
[0005] To solve the above problems, this application adopts the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides an interface element display method, which is applied to a terminal, and the method may include: the terminal displays a first interface of an application on a display screen of the terminal, and the first interface includes a first element; after receiving a drag operation on the first element, the terminal displays an animation of the first element moving with the drag operation on the display screen of the terminal; after receiving a drag and release operation, the terminal displays a first floating element on the display screen of the terminal, and the content of the first floating element is the same as the content of the first element.
[0007] With the above technical solution, the terminal can display any element on the interface as a floating element on the terminal screen according to the user's operation. Afterwards, no matter the user switches pages within the application or exits the application or opens other applications, the floating element can continue to be displayed on the terminal screen, which is used to realize cross-application information viewing, file transfer, information sharing and other operations.
[0008] In a possible implementation, the first interface also includes a second element within a preset range of the first element; the first floating element also includes the content of the second element. In this way, the predetermined elements in the first interface can be presented on the terminal screen in the form of floating elements according to user operations, further meeting the user's needs for cross-application information viewing, file transfer, information sharing, etc.
[0009] In another possible implementation, the first element is one or more of a text element, a uniform resource locator (URL), a picture element, an icon element, a card element, a list contact element, a video window element, or a file resource element.
[0010] Among them, when the first element is a text element, the content of the first floating element can be all or part of the text in the text element; when the first element is a URL, the content of the first floating element is a link icon, and the connection icon is used to link to the web page corresponding to the URL; when the first element is a picture element, the content of the first floating element is the picture in the picture element; when the first element is an icon element, the content of the first floating element is the icon in the icon element; when the first element is a list element, the content of the first floating element is the contact avatar included in the list element; when the first element is a file resource element, the content of the first floating element is a link icon, and the connection icon is used to link to the file resource.
[0011] In another possible implementation, after the terminal displays the first interface on the display screen, the method may further include: after receiving the first operation, the terminal displays the elements in the first interface that support floating display in a first form, and displays the elements that do not support floating display in a second form; the first element is an element that supports floating display, and the first form is different from the second form. In this way, the elements in the interface that support floating display can be highlighted, making it easier for users to check which elements support floating display, thereby improving user experience.
[0012] In another possible implementation manner, the first operation is an operation on a predetermined function control; or the first operation is an operation on an element in the first interface that supports floating display.
[0013] In another possible implementation, after the terminal displays the first floating element on the display screen of the terminal, the method may further include: the terminal displays a second interface on the display screen of the terminal, and the second interface includes a third element; after receiving a second operation on the third element, the terminal displays a second floating element on the display screen of the terminal, and the content of the second floating element is the same as the content of the third element, and the second floating element and the first floating element are displayed on the display screen of the terminal according to a preset arrangement rule. Among them, the second operation includes a drag operation and a drag-and-release operation, or when the third element is a text element or a URL address, the second operation can also be a copy operation. This technical solution supports the simultaneous display of multiple floating elements on the terminal screen to meet the needs of users. In addition, each floating element can be displayed according to sorting rules such as the order of generation time.
[0014] In another possible implementation, before the terminal displays the second floating element on the display screen of the terminal, the method may further include: the terminal determines that the number of floating elements displayed on the display screen of the terminal is less than N, where N is the maximum number of floating elements allowed to be displayed simultaneously on the display screen. The maximum number of floating elements allowed to be displayed simultaneously on the terminal screen can be limited, which not only meets user needs, but also does not affect the user experience due to too many floating elements in the interface.
[0015] In another possible implementation, the method may further include: when the terminal determines that the number of floating elements displayed on the display screen of the terminal is equal to N, deleting the floating UI element that is arranged first according to the preset arrangement rule among the N floating elements, and displaying the second floating element. In the case where an upper limit of floating elements allowed to be displayed simultaneously is set, when the number of floating elements on the current screen has reached the upper limit, the user triggers the generation of a new floating element, the earliest generated floating element can be deleted, and then the newly generated floating element can be displayed.
[0016] In another possible implementation, before the terminal displays the second floating element on the display screen of the terminal, the method may further include: the terminal determining that there is no floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal.
[0017] In another possible implementation, the method may further include: when the terminal determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal, the terminal merges the second floating element and the third floating element into a floating element group and displays it on the display screen of the terminal.
[0018] In another possible implementation manner, the method further includes: after receiving the third operation on the floating element group, the terminal displays the second floating element and the third floating element on a display screen of the terminal.
[0019] When a user triggers the generation of a new floating element, if there is an element of the same type as the newly generated floating element in the current interface, the newly generated floating element and the element can be automatically merged into a floating element group and displayed. If not, the new floating element is displayed. In this way, floating elements of the same type can be automatically classified and displayed, which improves the user experience. It also supports expanding the floating element group for users to view.
[0020] In another possible implementation, the method may further include: after receiving a fourth operation on the first floating element, the terminal deletes the first floating element; or, after the first floating element is displayed on the display screen of the terminal for a preset time, the terminal deletes the first floating element.
[0021] In another possible implementation, one or more of the following logos are displayed on the first floating element: an application logo, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element. In this way, it is convenient for the user to know which application the floating element comes from or its type.
[0022] In another possible implementation, when the first element is a text element, a URL, an image element or a file resource element, after the terminal displays the first floating element on the display screen, the method may also include: the terminal displays an interface including an input box; after the terminal receives a drag operation on the first floating element and releases the drag operation at the input box position, the terminal fills the content corresponding to the first floating element into the input box. For floating elements of text, image, card, file resource and video window types, the terminal can be triggered to send them to friends through an instant messaging application by dragging the floating element into the input box. The interactive process and operation steps when sharing information across applications are simplified, and the human-computer interaction performance is improved.
[0023] In another possible implementation, when the first element is a URL, after the terminal displays the first floating element on the display screen, the method may further include: after receiving the user's operation on the first floating element, the terminal displays the webpage corresponding to the URL. In this way, URL-type floating UI elements support direct click jump, which improves human-computer interaction performance.
[0024] In another possible implementation, when the first element is an icon element or a list contact element, after the terminal displays the first floating element on the display screen, the method may also include: after the terminal receives the fifth operation on the first floating element, the terminal executes the first function, and the first function is the same as the function executed after the terminal receives the user's operation on the first element. In this way, the floating elements representing specific contacts extracted from the contact lists of different applications support functions such as sending files and making calls according to specific business scenarios. The interaction process and operation steps when triggering the terminal to execute functions are simplified, and the human-computer interaction performance is improved.
[0025] In another possible implementation, when the first element is a list contact element, after the terminal displays the first floating element on the display screen, the method may also include: after the terminal receives a drag operation on the text, URL or picture in the interface and releases the drag operation at the position of the first floating element, the terminal sends the text, URL or picture to the terminal of the contact corresponding to the first floating element. Files, pictures, texts, URLs and other content can be directly shared with the corresponding contact by dragging them to the floating element of the list contact. The interaction process and operation steps when sharing information across applications are simplified, and the human-computer interaction performance is improved.
[0026] In some other embodiments, the terminal can also display the elements in the interface on the terminal screen in the form of floating elements through other operation triggers. For example, the terminal displays an interface on the display screen of the terminal; after receiving the screenshot operation of the user, the terminal captures the content of the interface displayed on the display screen and generates a screenshot; then, the terminal displays a floating element on the display screen of the terminal, and the floating element includes the screenshot. Alternatively, the terminal displays an interface on the display screen of the terminal; after receiving the screenshot operation of the user, the terminal captures the content of the interface displayed on the display screen, generates a screenshot and displays the screenshot; then, after receiving the user's drag operation on the screenshot and then releasing the drag operation, the terminal displays a floating element on the display screen, and the floating element includes the screenshot. For another example, an interface is displayed on the display screen of the terminal, and the interface includes a predetermined button; after receiving the operation of the predetermined button, the terminal displays a floating element on the display screen, and the content of the floating element is the same as the content of some elements in the interface.
[0027] In a second aspect, an embodiment of the present application provides an interface element display method, which is applied to a terminal. The method may include: the terminal displays a first interface on a display screen of the terminal, the first interface includes a first element, and the first element is a text element or a URL; after receiving a copy operation on the first element, the terminal displays a first floating element on the display screen of the terminal, and the content of the first floating element is the same as the content of the first element.
[0028] By adopting the above technical solution, the terminal can display the text elements or URL addresses on the interface as floating elements on the terminal screen according to the user operation. After that, no matter whether the user switches pages within the application or exits the application or opens other applications, the floating element can continue to be displayed on the terminal screen, which is used to realize cross-application information viewing, file transfer, information sharing and other operations. In addition, combined with the existing interactive logic of the system, if the terminal is triggered to display a copy button for the element, the user can trigger the text element or URL address to be displayed on the terminal screen as a floating element by clicking the copy button, thereby improving the user experience.
[0029] In a possible implementation, after the terminal displays the first floating element on the display screen, the method may further include: the terminal displays an interface including an input box; after the terminal receives a drag operation on the first floating element and releases the drag operation at the input box position, the terminal fills the content corresponding to the first floating element into the input box. This simplifies the interaction process and operation steps when sharing information across applications, and improves human-computer interaction performance.
[0030] In a third aspect, an embodiment of the present application provides a display device, which can be applied to a terminal, and the device may include: a display unit, used to display a first interface of an application on a display screen of the terminal, the first interface including a first element; an input unit, used to receive a drag operation on the first element; the display unit is also used to display an animation of the first element moving with the drag operation on the display screen of the terminal; the input unit is also used to receive a drag and release operation; the display unit is also used to display a first floating element on the display screen of the terminal, and the content of the first floating element is the same as the content of the first element.
[0031] In a possible implementation, the first interface also includes a second element within a preset range of the first element; and the first floating element also includes content of the second element.
[0032] In another possible implementation, the first element is one or more of a text element, a URL, a picture element, an icon element, a card element, a list contact element, a video window element, or a file resource element.
[0033] In another possible implementation, the input unit is further used to receive the first operation; the display unit is further used to display the elements in the first interface that support floating display in the first form, and display the elements that do not support floating display in the second form; the first element is an element that supports floating display, and the first form is different from the second form.
[0034] In another possible implementation manner, the first operation is an operation on a predetermined function control; or the first operation is an operation on an element in the first interface that supports floating display.
[0035] In another possible implementation, the display unit is further used to display a second interface on the display screen of the terminal, and the second interface includes a third element; the input unit is further used to receive a second operation on the third element; the display unit is further used to display a second floating element on the display screen of the terminal, the content of the second floating element is the same as the content of the third element, and the second floating element and the first floating element are displayed on the display screen of the terminal according to a preset arrangement rule. The second operation may include a drag operation and a drag-and-release operation, or when the first element is a text element or a URL, the second operation may be a copy operation.
[0036] In another possible implementation, the display device may further include: a determining unit, configured to determine that the number of floating elements displayed on the display screen of the terminal is less than N, where N is the maximum number of floating elements allowed to be displayed simultaneously on the display screen.
[0037] In another possible implementation, the display device may also include: a deletion unit, which is used to delete the first floating UI element arranged according to a preset arrangement rule among the N floating elements when the determination unit determines that the number of floating elements displayed on the display screen of the terminal is equal to N, and display the second floating element.
[0038] In another possible implementation manner, the determining unit is further configured to determine that no floating element of the same type as the second floating element exists among the floating elements displayed on the display screen of the terminal.
[0039] In another possible implementation, the display unit is also used to merge the second floating element and the third floating element into a floating element group and display it on the display screen of the terminal when the determination unit determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal.
[0040] In another possible implementation, the input unit is further configured to receive a third operation on the floating element group; and the display unit is configured to display the second floating element and the third floating element on a display screen of the terminal.
[0041] In another possible implementation, the input unit is further used to receive a fourth operation on the first floating element, and the deletion unit is further used to delete the first floating element; or, the deletion unit is further used to delete the first floating element after the first floating element is displayed on the display screen of the terminal for a preset period of time.
[0042] In another possible implementation, one or more of the following identifiers are displayed on the first floating element: an application identifier, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element.
[0043] In another possible implementation, when the first element is a text element, a URL, an image element or a file resource element, the display unit is also used to display an interface including an input box; the input unit is also used to receive a drag operation on the first floating element and release the drag operation at the input box position; the display device may also include: a filling unit, used to fill the content corresponding to the first floating element into the input box.
[0044] In another possible implementation, when the first element is an icon-type element or a list-contact human element, the input unit is also used to receive a fifth operation on the first floating element; the display device may also include: an execution unit, used to execute a first function, and the first function is the same as the function executed after the terminal receives the user's operation on the first element.
[0045] In another possible implementation, when the first element is a list contact element, the input unit is also used to receive a drag operation on text, URL or image in the interface, and release the drag operation at the position of the first floating element; the display device may also include: a sending unit, used to send the text, URL or image to the terminal of the contact corresponding to the first floating element.
[0046] In a fourth aspect, an embodiment of the present application provides a display device, which is applied to a terminal. The display device may include: a display unit, which is used to display a first interface on a display screen of the terminal, where the first interface includes a first element, and the first element is a text element or a uniform resource locator (URL); an input unit, which is used to receive a copy operation on the first element; and the display unit, which is also used to display a first floating element on the display screen of the terminal, where the content of the first floating element is the same as the content of the first element.
[0047] In one possible implementation, the display unit is also used to display an interface including an input box; the input unit is also used to receive a drag operation on the first floating element and release the drag operation at the input box position; the display device may also include: a filling unit, used to fill the content corresponding to the first floating element into the input box.
[0048] In a fifth aspect, an embodiment of the present application provides a display device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute the instructions so that the display device implements a method as described in the first aspect or any possible implementation of the first aspect, or implements a method as described in the second aspect or any possible implementation of the second aspect.
[0049] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by an electronic device, the electronic device implements a method as described in the first aspect or any possible implementation of the first aspect, or implements a method as described in the second aspect or any possible implementation of the second aspect.
[0050] In the seventh aspect, an embodiment of the present application provides an electronic device (such as the above-mentioned terminal), which includes a display screen, one or more processors and a memory; the display screen, the processor and the memory are coupled; the memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the electronic device, the electronic device executes a method as described in the first aspect or any possible implementation of the first aspect, or executes a method as described in the second aspect or any possible implementation of the second aspect.
[0051] In an eighth aspect, an embodiment of the present application provides a computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code is executed in an electronic device (such as the above-mentioned terminal), a processor in the electronic device executes a method as described in the first aspect or any possible implementation of the first aspect, or executes a method as described in the second aspect or any possible implementation of the second aspect.
[0052] It can be understood that the beneficial effects that can be achieved by the display device described in the third aspect and any possible implementation thereof, the display device described in the fourth aspect and any possible implementation thereof, the display device described in the fifth aspect, the computer-readable storage medium described in the sixth aspect, the electronic device described in the seventh aspect, and the computer program product described in the eighth aspect can be referred to as the beneficial effects in the first aspect and any possible implementation thereof or the second aspect and any possible implementation thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 A schematic diagram of a display interface in a related technology provided in an embodiment of the present application;
[0054] Figure 2 A schematic diagram of a display interface in another related technology provided in an embodiment of the present application;
[0055] Figure 3 A schematic diagram of the structure of a mobile phone provided in an embodiment of the present application;
[0056] Figure 4 A schematic diagram of the composition of a software architecture of a terminal provided in an embodiment of the present application;
[0057] Figure 5A A schematic diagram of a display interface provided in an embodiment of the present application;
[0058] Figure 5B A schematic diagram of another display interface provided in an embodiment of the present application;
[0059] Figure 6 A flowchart of a method for displaying interface elements provided in an embodiment of the present application;
[0060] Figure 7 A schematic diagram of another display interface provided in an embodiment of the present application;
[0061] Figure 8 A schematic diagram of another display interface provided in an embodiment of the present application;
[0062] Fig. 9 A schematic diagram of another display interface provided in an embodiment of the present application;
[0063] Fig.10 A schematic diagram of another display interface provided in an embodiment of the present application;
[0064] Fig.11 A schematic diagram of another display interface provided in an embodiment of the present application;
[0065] Fig.12 A flowchart of another method for displaying interface elements provided in an embodiment of the present application;
[0066] Fig.13A A schematic diagram of another display interface provided in an embodiment of the present application;
[0067] Fig. 13B A schematic diagram of another display interface provided in an embodiment of the present application;
[0068] Fig.14 A schematic diagram of another display interface provided in an embodiment of the present application;
[0069] Fig.15 A schematic diagram of another display interface provided in an embodiment of the present application;
[0070] Fig.16 A schematic diagram of another display interface provided in an embodiment of the present application;
[0071] Fig.17 A schematic diagram of another display interface provided in an embodiment of the present application;
[0072] Fig.18 A schematic diagram of another display interface provided in an embodiment of the present application;
[0073] Fig.19 A schematic diagram of another display interface provided in an embodiment of the present application;
[0074] Fig. 20 A schematic diagram of another display interface provided in an embodiment of the present application;
[0075] Fig.21 A schematic diagram of another display interface provided in an embodiment of the present application;
[0076] Fig. 22 A schematic diagram of another display interface provided in an embodiment of the present application;
[0077] Fig.23 A schematic diagram of another display interface provided in an embodiment of the present application;
[0078] Fig.24 A schematic diagram of another display interface provided in an embodiment of the present application;
[0079] Fig.25 A schematic diagram of another display interface provided in an embodiment of the present application;
[0080] Fig.26 A schematic diagram of another display interface provided in an embodiment of the present application;
[0081] Fig. 27 A schematic diagram of another display interface provided in an embodiment of the present application;
[0082] Fig.28 A schematic diagram of another display interface provided in an embodiment of the present application;
[0083] Fig.29 A schematic diagram of another display interface provided in an embodiment of the present application;
[0084] Fig.30 A schematic diagram of another display interface provided in an embodiment of the present application;
[0085] Fig.31 A schematic diagram of another display interface provided in an embodiment of the present application;
[0086] Fig.32 A schematic diagram of the composition of a display device provided in an embodiment of the present application;
[0087] Fig.33 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0088] At present, by installing applications in mobile phones and other terminals, the functions of mobile phones can be improved to provide users with a better user experience. However, in the process of daily use of mobile phones, the following scenarios are often encountered: to share a picture of a product with a friend, you need to save it to the gallery application first, and then click on the picture from the gallery application to complete the sharing, which is very troublesome to interact; if the content such as the phone number shared by a friend is shared through a picture, it cannot be copied directly. If it is copied after recognition using optical character recognition (OCR), the OCR recognition is inaccurate, and the user can only switch back and forth between the typing interface and the picture interface on the mobile phone; send a website URL to a friend, after copying the URL, the copied content is only saved in the clipboard, which is not visible to the user, and it is necessary to switch multiple applications to share it with friends, which is a poor experience; after purchasing a product, it is necessary to go through multiple operation steps to check whether the product has been shipped, the logistics of the product, and other information, and the interaction process is long. These scenarios all reflect that when users use mobile phones and other terminals in daily life, if they need to view or share information across applications, there will be problems such as cumbersome interaction processes, many operation steps, and poor human-computer interaction performance.
[0089] Current technology provides some solutions to solve the above problems.
[0090] Related technology 1: Huawei customized the sidebar (dock) function. Users can trigger the phone to display the dock bar by sliding from the left or right edge of the phone screen to the middle area of the phone screen. The dock bar cannot be moved on the phone screen. When the user clicks other areas outside the dock bar, the dock bar is closed. The dock bar includes one or more application icons, providing users with a quick entry to quickly open the corresponding application, thereby simplifying the user's interaction process when using multiple applications and improving usage efficiency. For example, combined with Figure 1 ,like Figure 1 As shown in (a) of FIG. 1 , the mobile phone displays a memo interface 101. The user wants to send the content in the memo to WeChat. TM Share with friends. The user can perform a sliding operation on the mobile phone screen from the right edge of the screen to the middle area of the mobile phone screen. In response to the sliding operation, the mobile phone can display a dock bar 102, which includes one or more application icons, such as WeChat. TM Icon 103. When the user clicks on the WeChat icon in the dock bar 102 TM The icon 103 is operated, such as after clicking the operation, Figure 1 As shown in (b), the mobile phone can display WeChat in the form of a freeform small window 104 (or floating window). TM After that, users can TMSelect a friend in the interface to share the content through WeChat TM However, the dock only provides a quick entry to open the application, and users still need to go through more cumbersome steps to share content with friends, and the human-computer interaction performance has not been improved.
[0091] Related technology 2: WeChat TM Defines the floating window (or favorites bar) function. The floating window is the entry for temporarily storing links to mini programs or articles. For example, combined with Figure 2 ,like Figure 2 As shown in (a), the user browses WeChat TM The user can perform a sliding operation from the left side to the right side of the mobile phone screen to trigger the mobile phone to return to the previous page of the information page 201. Figure 2 As shown in (b) of FIG. 1 , as the user slides the finger, the mobile phone can display a partial area of the information page 201 and a partial area 202 of the upper level page of the information page 201. In response to the user's sliding operation, the mobile phone can also display a floating window indication area 203 in the lower right corner of the mobile phone screen. After the user slides the finger into the floating window indication area 203, as shown in FIG. Figure 2 As shown in (c) of FIG. 1 , the mobile phone displays a floating window 204 in the form of a favorites drawer, and after the user releases the hand, the floating window 204 is automatically adsorbed to the edge of the screen. The floating window 204 includes a link to the information page 201, that is, the information page 201 is included in the favorites bar, and the user clicks the favorites bar to view the included information page 201. Among them, WeChat TM A floating window can temporarily store (or favorite) up to five mini-programs or article links at the same time. When multiple mini-programs or article links are favorited in a floating window, the user can view the favorite list by clicking on the floating window. TM It keeps showing when WeChat TM After exiting, the floating window will no longer be displayed. TM After exiting, the floating window will not be able to serve as a quick entry, so it cannot solve the problem of poor human-computer interaction when viewing or sharing information across applications. TM The floating window function can only temporarily store the link of the entire page in the form of a floating window, and cannot support the temporary storage of some or a certain element in the page in the form of a floating window.
[0092] Related technology 3: Android TM The bubble function is provided. Applications (such as instant messaging applications) can call the corresponding interface of this function and, after receiving corresponding user operations (such as long pressing or swiping the contact list to the left, clicking the More button, and then selecting to open the floating chat avatar), the contact avatar in the application can be displayed on the edge of the mobile phone screen in the form of a bubble.TM The bubble function provided is only used to provide a quick entry to the contact's chat interface. That is, after the user clicks on the contact's avatar displayed in the form of a bubble, the phone can display the contact's chat interface, but has no other functions. It does not support the display of other content in the form of bubbles except the contact's avatar. In addition, there are many steps to trigger the display of the contact's avatar in the form of bubbles, and the interaction is complex, so the human-computer interaction performance cannot be well improved.
[0093] It can be seen that the solutions provided by the above related technologies cannot effectively solve the problem of poor human-computer interaction performance when viewing or sharing information across applications.
[0094] The embodiment of the present application provides a method for displaying interface elements, which can be applied to a terminal. Using the method provided in this embodiment, after receiving the user's drag, copy and other operations on any element in the interface, the terminal can convert the element into a floating user interface (UI) element and display it on the terminal screen, such as adsorption display on the edge of the terminal screen. In addition, whether the user switches pages in the application or exits the application, the floating UI element can continue to be displayed on the terminal screen and displayed on the top layer of the interface. The floating UI element in this embodiment can be moved, supports interaction with other interface elements, opens the corresponding interface after being clicked, receives the content dragged by the user and sends it out, and other functions. In this way, it can help users to realize local cross-application file transfer, information sharing and other operations in a more convenient interactive way. The interaction process during cross-application interaction is simplified, the operation steps are reduced, and the human-computer interaction performance is improved.
[0095] Exemplarily, the terminal in the embodiment of the present application can be a mobile phone, a tablet computer, a handheld computer, a PC, a cellular phone, a personal digital assistant (PDA), a wearable device (such as a smart watch), a car computer, a game console, and an augmented reality (AR) / virtual reality (VR) device, etc. This embodiment does not impose any special restrictions on the specific form of the terminal. In addition, the technical solution provided in this embodiment can be applied to the above-mentioned terminals (or mobile terminals) as well as other electronic devices, such as smart home devices (such as televisions).
[0096] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0097] In this embodiment, the terminal is taken as a mobile phone. Figure 3 , is a schematic diagram of the structure of a mobile phone provided in an embodiment of the present application. The methods in the following embodiments can be implemented in a mobile phone having the above hardware structure.
[0098] like Figure 3 As shown, the mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a display screen 194. Optionally, the mobile phone may also include a mobile communication module 150, a subscriber identification module (SIM) card interface 195, and the like.
[0099] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0100] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone. In other embodiments, the mobile phone may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0101] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0102] The controller can be the nerve center and command center of the mobile phone. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.
[0103] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0104] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.
[0105] The charging management module 140 is used to receive charging input from the charger. While the charging management module 140 is charging the battery 142, it can also power the mobile phone through the power management module 141. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 can also receive input from the battery 142 to power the mobile phone.
[0106] The wireless communication function of the mobile phone can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.
[0107] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0108] When the mobile phone includes a mobile communication module 150, the mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the mobile phone. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.
[0109] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.
[0110] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), NFC, infrared technology (IR), etc., which are applied to mobile phones. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0111] In some embodiments, the antenna 1 of the mobile phone is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite based augmentation system (SBAS).
[0112] The mobile phone realizes the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
[0113] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include 1 or N display screens 194, where N is a positive integer greater than 1.
[0114] The mobile phone can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor, etc. In some embodiments, the mobile phone can include 1 or N cameras 193, where N is a positive integer greater than 1.
[0115] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.
[0116] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
[0117] The mobile phone can realize audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.
[0118] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. When a touch operation is applied to the display screen 194, the mobile phone detects the intensity of the touch operation according to the pressure sensor 180A. The mobile phone can also calculate the touch position according to the detection signal of the pressure sensor 180A.
[0119] The gyroscope sensor 180B can be used to determine the motion posture of the mobile phone. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The mobile phone can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. The acceleration sensor 180E can detect the magnitude of the acceleration of the mobile phone in all directions (generally three axes). The distance sensor 180F is used to measure the distance. The mobile phone can use the proximity light sensor 180G to detect that the user holds the mobile phone close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used for automatic unlocking and locking of the leather case mode and pocket mode. The ambient light sensor 180L is used to sense the brightness of the ambient light. The fingerprint sensor 180H is used to collect fingerprints. The mobile phone can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc. The temperature sensor 180J is used to detect temperature.
[0120] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the mobile phone, which is different from the position of the display screen 194.
[0121] The bone conduction sensor 180M can obtain vibration signals. The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.
[0122] When the mobile phone includes a SIM card interface 195, the SIM card interface 195 is used to connect the SIM card. The SIM card can be connected to and separated from the mobile phone by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The mobile phone can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The mobile phone interacts with the network through the SIM card to achieve functions such as calls and data communications. In some embodiments, the mobile phone uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the mobile phone and cannot be separated from the mobile phone.
[0123] This application embodiment takes the terminal software system as an example of the Android system to illustrate the software structure of the terminal. Figure 4 , which is a schematic diagram of the composition of a software architecture of a terminal provided in an embodiment of the present application.
[0124] Among them, the terminal software system can adopt a layered architecture, event-driven architecture, micro-kernel architecture, microservice architecture, or cloud architecture. Take the terminal software system as an example. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces.
[0125] In some embodiments, Figure 4 As shown, the terminal may include an application layer and a framework layer (framework, FWK). The application layer may include a series of application packages. For example, the application package may include settings, calculator, camera, navigation, shopping, short message, video player, music player, phone and other applications. The applications included in the application layer may be system applications of the terminal or third-party applications, which are not specifically limited in this embodiment of the present application. The application layer may also include a drag service module and an element control module (controller). This embodiment is not specifically limited here.
[0126] The framework layer is mainly responsible for providing application programming interface (API) and programming framework for the application layer. The framework layer may include a window manager (WM), etc. The management manager may also be called a window management module, which is not specifically limited in this embodiment.
[0127] Of course, the terminal can also include other layers, such as the kernel layer ( Figure 4 The kernel layer is a layer between hardware and software. The kernel layer may at least include display driver, camera driver, audio driver, sensor driver, etc.
[0128] In the embodiment of the present application, based on the above software architecture, when the terminal currently displays the interface of a certain application, after receiving the user's drag or copy operation on any element in the interface, the element can be converted into a floating UI element and displayed on the terminal screen, such as being adsorbed and displayed on the edge of the terminal screen. Afterwards, no matter the user switches pages within the application or exits the application or opens other applications, the floating UI element can continue to be displayed on the terminal screen, which is used to realize cross-application information viewing, file transfer, information sharing and other operations.
[0129] The following takes a mobile phone as an example, and describes in detail the interface element display method provided by the embodiment of the present application in combination with the accompanying drawings. In this embodiment, the interface element that supports the display of floating UI elements can be any element in the interface. For example, it can be a text class, a picture class, an icon class, a card class, a list contact class, a video window class, a file resource class, and other interface elements.
[0130] The following is an example of how different types of interface elements are displayed as floating UI elements on a mobile phone screen.
[0131] Combination Figure 5A , Figure 5B and Figure 6 , an example is given to illustrate the process of displaying text elements on a mobile phone screen in the form of floating UI elements.
[0132] The operation that triggers the text element to be displayed in the form of a floating UI element may be a drag operation of the text element by the user, or may be a copy operation of the text element.
[0133] For example, combined with Figure 5A For example, the operation that triggers the display of a text element as a floating UI element is a user's dragging operation on the text element. Figure 5A As shown in (a) of FIG. 5 , the mobile phone currently displays an information page 501 of a certain application. When the user wants to display certain text in the information page 501 on the mobile phone screen in the form of a floating UI element, the user can perform a drag operation on the text. After the user releases the drag, the mobile phone can display the text on the mobile phone screen in the form of a floating UI element.
[0134] The above drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the text to float and an operation of triggering the floating text to move.
[0135] For example, the operation that triggers the text to float is the operation of long pressing and selecting the text, and the operation that triggers the floating text to move is the operation of pressing and moving the floating text. Figure 5AAs shown in (a) of FIG. 5 , the user can long press and select text 502 in information page 501. Information page 501 can be the first interface in the embodiment of the present application. Text 502 can be the first element in the embodiment of the present application. The mobile phone can receive the user's long press and selection operation on text 502. In response to the long press and selection operation, the mobile phone can display text 502 in a floating state. For example, Figure 5A As shown in (b) of FIG. 1 , the mobile phone can perform background blur processing and display floating text 502. Afterwards, the user presses the floating text 502 and moves the finger, so that the text 502 moves on the mobile phone screen along with the movement of the user's finger. Figure 5A As shown in (b) of FIG. 1 , the user can press the floating text 502 with a finger and move the finger, such as moving the finger to the right. In response to the user's operation, the mobile phone can display an animation of the text 502 moving with the user's finger, giving the user a visual effect that the text 502 is being dragged by the user. Afterwards, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in FIG. Figure 5A As shown in (c) of FIG. 1 , the mobile phone can display a floating UI element 503 on the mobile phone screen. The content of the floating UI element 503 is the same as the text 502, that is, the text selected and dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. For example, the floating UI element 503 can be adsorbed and displayed on the edge of the mobile phone screen (such as the left edge of the screen or the right edge of the screen, Figure 5A (c) in the figure is shown by taking adsorption display on the right edge of the screen as an example). The floating UI element 503 may be the first floating element in the embodiment of the present application.
[0136] For example, combining Figure 5B , taking the operation of triggering the text element to be displayed as a floating UI element as an example, the operation of copying the text element. Figure 5B As shown in (a) of FIG. 1 , the mobile phone currently displays an information page 601 of a certain application. When the user wants to display certain text in the information page 601 on the mobile phone screen in the form of a floating UI element, the user can perform a copy operation on the text. After receiving the copy operation, the mobile phone can display the text on the mobile phone screen in the form of a floating UI element.
[0137] The copy operation for text may include one or more operations. For example, the copy operation for text may include a long press operation on the text and an operation on a copy button. Figure 5B As shown in (a) of FIG. 1 , the user can long press the text 602 in the information page 601. The information page 601 can be the first interface in the embodiment of the present application. The text 602 can be the first element in the embodiment of the present application. Figure 5BAs shown in (b) of FIG. 5 , in response to a user's long press operation on text 602, the mobile phone may display an operation menu, which includes a copy button 603. After receiving the user's operation on the copy button 603, such as a click operation, the mobile phone Figure 5B As shown in (c) of FIG. 1 , the mobile phone can display a floating UI element 604 on the mobile phone screen. The content of the floating UI element 604 is the same as the text 602, that is, the text copied by the user is displayed on the mobile phone screen in the form of a floating UI element. The floating UI element 604 can be the first floating element in the embodiment of the present application.
[0138] The drag operation and the drag-and-release operation may be the second operation in the embodiment of the present application. The copy operation may be the second operation in the embodiment of the present application.
[0139] It should be noted that the above Figure 5A and Figure 5B The illustrated embodiment is described by taking the example that the text element is the text contained in the information page of the application. The application can be a system application of the mobile phone or a third-party application, which is not limited in this embodiment. In addition, the text element can also be the text contained in other pages, such as the chat content in the chat interface of the instant messaging application, and the text contained in the page of the shopping application, which is not specifically limited in this embodiment.
[0140] Combination Figure 4 ,like Figure 6As shown, as an example, the specific implementation process of presenting text-type elements as floating UI elements on the mobile phone screen can be: after the mobile phone, such as the application of the mobile phone application layer, receives the user's drag operation on the text, it can trigger the dragging of the text, and obtain the drag event and send it to the drag service module of the mobile phone application layer. For example, the process of the application of the mobile phone application layer obtaining the drag event can be: after the user performs the drag operation, the framework layer of the mobile phone can monitor the drag operation (such as a long press operation on the floating text), and in response, the framework layer of the mobile phone can generate a drag event (DragEvent). The application of the mobile phone application layer can obtain the drag event generated by the framework layer through callback. Among them, the drag event may include the content of the dragged text and the type of the dragged element (such as text class). After receiving the drag event, the drag service module of the mobile phone application layer can cooperate with the element control module of the mobile phone application layer to construct a shadow according to the type of the dragged element and the content of the text in the drag event. If the type of the dragged element is text, the drag service module of the mobile application layer can transmit the content of the text included in the drag event to the element control module, and the element control module can generate a shadow based on the content of the text. Afterwards, the element control module of the mobile application layer can transmit the generated shadow to the window manager of the mobile phone framework layer, so that the window manager displays the shadow in the form of a floating UI element. For example, the window manager displays a rounded text box on the mobile phone screen, and the rounded text box is a floating UI element, which includes the text dragged by the user. Similarly, after the application of the mobile application layer receives the user's copy operation of the text, it can also trigger the framework layer of the mobile phone to send the content of the copied text to the drag service module of the mobile application layer, so that the drag service module cooperates with the element control module to construct the shadow, and the window manager of the mobile phone framework layer displays the shadow in the form of a floating UI element, thereby realizing the presentation of text-type elements on the mobile phone screen in the form of floating UI elements.
[0141] Use the above Figure 5A or Figure 5BThe technical solution of the illustrated embodiment, for text-based interface elements, the user can display the corresponding text on the mobile phone screen in the form of a floating UI element by performing a drag operation or a copy operation, such as adsorption display on the edge of the terminal screen. That is, the text dragged or copied by the user can be explicitly presented on the mobile phone screen through the floating UI element. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view information across applications. And the floating UI element can be moved. For example, the user can perform a drag operation on the floating UI element, such as pressing and moving a finger at the display position of the floating UI element. In response to the user's drag operation, the mobile phone can display an animation of the floating UI element moving on the mobile phone screen. In addition, when the user wants to use the text across pages or across applications, such as entering the text into the input box of other applications, there is no need to switch pages or applications back and forth. By dragging the floating UI element into the input box, the text in the floating UI element can be filled into the input box. Specifically, as an example, after the mobile phone displays text in the form of a floating UI element on its screen, the mobile phone (such as the framework layer of the mobile phone) can associate the identifier of the floating UI element with the content of the text and store it in the mobile phone. After that, when the mobile phone displays a certain application interface including an input box, after receiving the user's drag operation on the floating UI element, it can display the animation of the floating UI element moving. After the user drags the floating UI element to the display position of the input box, hovers and releases the finger, in response to the operation, the mobile phone can fill the text in the floating UI element into the input box. For example, after the user drags the floating UI element to the display position of the input box and hovers, the element control module of the mobile phone application layer can notify the framework layer of the mobile phone to send the data corresponding to the floating UI element, that is, the content of the text corresponding to the identifier of the floating UI element to the application. After that, after the user releases his finger, that is, releases the drag, the application of the mobile phone application layer can receive the content of the text sent by the mobile phone framework layer and fill the text into the input box.
[0142] Combination Figure 7 and Figure 8 , an example is given to illustrate the process of displaying picture elements on the mobile phone screen in the form of floating UI elements.
[0143] The operation of triggering the display of the picture element in the form of a floating UI element may be a dragging operation of the picture element by the user.
[0144] For example, combined with Figure 7 ,like Figure 7As shown in (a) in the figure, the mobile phone displays an interface 701 of the gallery application. The interface 701 includes multiple pictures. The interface 701 may be the first interface in the embodiment of the present application. When the user wants to display a picture in the interface 701 on the mobile phone screen as a floating UI element, the user may perform a drag operation on the picture. After the user releases the drag, the mobile phone may display the picture (such as a thumbnail of the picture) on the mobile phone screen as a floating UI element.
[0145] The drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the picture to float and an operation of triggering the floating picture to move.
[0146] For example, the operation that triggers the image to float is a long press operation on the image, and the operation that triggers the floating image to move is an operation that presses and moves the floating image. Figure 7 As shown in (a) of FIG. 1 , the user can long press the picture 702 in the interface 701 of the gallery application. The picture 702 can be the first element in the embodiment of the present application. The mobile phone can receive the user's long press operation on the picture 702. In response to the long press operation, the mobile phone can display the picture 702 in a floating state. For example, Figure 7 As shown in (b) of FIG. 1 , the mobile phone can perform background blur processing and display a floating picture 702. Afterwards, the user presses the floating picture 702 and moves the finger, so that the picture 702 moves on the mobile phone screen along with the movement of the user's finger. Figure 7 As shown in (b) of FIG. 1 , the user can press the floating picture 702 with a finger and move the finger, such as moving the finger to the right. In response to the user's operation, the mobile phone can display an animation of the picture 702 moving with the user's finger, giving the user a visual effect that the picture 702 is being dragged by the user. Afterwards, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in FIG. Figure 7 As shown in (c) of FIG. 1 , the mobile phone can display a floating UI element 703 on the mobile phone screen. The floating UI element 703 includes a picture 702, such as a thumbnail of the picture 702, that is, the picture dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. For example, the floating UI element 703 can be adsorbed and displayed on the edge of the mobile phone screen (such as the left edge of the screen or the right edge of the screen, Figure 7 (c) in FIG. 1 is shown as an example of adsorption display on the right edge of the screen). The floating UI element 703 may be the first floating element in the embodiment of the present application. The drag operation and the drag-and-release operation may be the second operation in the embodiment of the present application.
[0147] For example, combining Figure 8After the user takes a screenshot, the generated screenshot can be displayed on the mobile phone screen in the form of a floating UI element.
[0148] Among them, the user triggers the mobile phone to take a screenshot, such as the so-called screenshot operation, which can be an operation on the screenshot button (such as a click operation), or an operation on the function key of the mobile phone (such as the volume control button: volume "+" key, volume "-" key, switch button, etc.) or a function combination key (such as the combination of the volume "+" key and the switch button), or an input preset gesture, such as knuckle tapping, etc. The content you want to capture can be all the content currently displayed on the mobile phone screen, or part of the content. If the content you want to capture is part of the content currently displayed on the mobile phone screen, the above screenshot operation can also include the operation of selecting the capture area by drawing a circle or a frame. After the mobile phone receives the screenshot operation, the mobile phone can start the screenshot function to capture all or part of the content currently displayed on the mobile phone screen, thereby generating a screenshot. After the screenshot is generated, the mobile phone can display the generated screenshot on the mobile phone screen in the form of a floating UI element.
[0149] For example, combined Figure 8 , take the screenshot operation as an example of the operation on the screenshot button. Figure 8 As shown in (a) of FIG. 8 , the mobile phone displays the desktop 801. When the user wants to use the screenshot function, the user can perform a sliding operation from the top edge of the mobile phone screen to the bottom edge. The mobile phone can receive the sliding operation. In response to the sliding operation, as shown in FIG. Figure 8 As shown in (b) in the figure, the mobile phone displays a menu bar 802, and the menu bar 802 includes a screenshot button 803. The user can operate the screenshot button 803, such as clicking it. After the mobile phone receives the user's click operation on the screenshot button 803, it can start the screenshot function to capture the content of the desktop 801 currently displayed on the mobile phone screen, thereby generating a screenshot. After the screenshot is generated, the mobile phone can display the generated screenshot on the mobile phone screen in the form of a floating UI element. As an example, the mobile phone can first display the generated screenshot on the mobile phone screen in the form of a floating window, and then the screenshot becomes a floating UI element with a gradually shrinking animation (such as Figure 8 The floating UI element 805 shown in (d) is then adsorbed and displayed on the edge of the mobile phone screen. As another example, Figure 8 As shown in (c) in FIG. 1 , the mobile phone may first display the generated screenshot in the form of a floating window on the mobile phone screen, such as displaying screenshot 804. The user may press screenshot 804 and move the finger (such as moving the finger to the right) so that screenshot 804 moves on the mobile phone screen following the movement of the user's finger. Afterwards, the user may release the finger to release the drag. After the user releases the finger, the mobile phone may receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in FIG. Figure 8As shown in (d) of FIG. 1 , the mobile phone can display a floating UI element 805 on the mobile phone screen. The floating UI element 805 includes a screenshot 804, such as a thumbnail of the screenshot 804, that is, the generated screenshot is displayed on the mobile phone screen in the form of a floating UI element. For example, the floating UI element 805 can be displayed on the edge of the mobile phone screen (such as the left edge of the screen or the right edge of the screen, Figure 8 (d) in the figure is shown as an example of adsorption display on the right edge of the screen).
[0150] It should be noted that the above Figure 7 The illustrated embodiment is described by taking the picture element as a picture in a gallery application as an example. The picture element may also be a picture contained in a page of other mobile phone applications, such as a picture shared with friends or shared by friends in a chat interface of an instant messaging application, or a picture contained in a browser page, etc., and this embodiment does not make any specific restrictions here.
[0151] In addition, the specific implementation process of presenting picture elements on the mobile phone screen as floating UI elements is similar to Figure 6 The specific implementation process of presenting the text element as a floating UI element on the mobile phone screen is similar and will not be described here. The difference is that the drag event includes the dragged image and the type of the dragged element (such as the image class), and the element control module of the mobile phone application layer can generate a shadow based on the image.
[0152] Use the above Figure 7 or Figure 8The technical solution of the illustrated embodiment, for the interface elements of the picture type, the user can display the corresponding picture on the mobile phone screen in the form of a floating UI element by performing a drag operation or a screenshot operation, such as adsorption display on the edge of the terminal screen. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view or share information across applications. For example, after the mobile phone receives the user's click operation on the floating UI element, the mobile phone can call the system's application selector for the user to choose which application to use to view the picture displayed in the form of a floating UI element. For example, after the mobile phone displays the picture in the form of a floating UI element on its screen, the mobile phone (such as the frame layer of the mobile phone) can associate the identifier of the floating UI element with the picture and store it in the mobile phone. After the user selects the application to view the picture, the application can display the picture in the floating UI element to the user through the application according to the stored association relationship. And the floating UI element can be moved. For another example, when a user wants to use the picture across pages or applications, such as sharing the picture with a friend, there is no need to open the gallery application again to view and select. By dragging the floating UI element to the input box of the chat interface with the friend, the picture in the floating UI element can be shared with the friend, etc., which improves the interaction efficiency. The specific implementation is similar to the implementation of filling the text displayed in the form of a floating UI element into the input box, which will not be described in detail here.
[0153] Combination Fig. 9 , an example is given to illustrate the process of displaying a uniform resource locator (URL) on a mobile phone screen in the form of a floating UI element.
[0154] The process of displaying a URL on a mobile phone screen as a floating UI element is similar to the process of displaying a text element on a mobile phone screen as a floating UI element. The detailed process will not be repeated here. Here, an example is used to illustrate the process of displaying a URL on a mobile phone screen as a floating UI element. Figure 5A and Figure 5B In the description of the embodiment shown, the operation to trigger the URL to be displayed in the form of a floating UI element can be a drag operation of the user on the URL, or a copy operation on the URL. Fig. 9 For example, the operation that triggers the URL to be displayed in the form of a floating UI element is a copy operation of the URL. Fig. 9As shown in (a), the mobile phone currently displays web page 901. When the user wants to display the URL of web page 901 on the mobile phone screen in the form of a floating UI element, the user can perform a copy operation on the URL. After receiving the copy operation, the mobile phone can display the URL on the mobile phone screen in the form of a floating UI element. The copy operation can be the second operation in the embodiment of the present application.
[0155] The copy operation for the URL address may include one or more operations. Exemplarily, the copy operation for the URL address may include a long press operation on the URL bar (or the URL address in the URL bar) and an operation on the copy button. Fig. 9 As shown in (a) in FIG. 1 , the user can long press the URL bar 902 in the web page 901. The web page 901 can be the first interface in the embodiment of the present application. The URL in the address bar 902 can be the first element in the embodiment of the present application. Fig. 9 As shown in (b) of FIG. 1 , in response to a user's long press operation on the URL bar 902, the mobile phone may display an operation menu, which includes a copy button 903. After receiving the user's operation on the copy button 903, such as a click operation, the mobile phone Fig. 9 As shown in (c) of FIG. 1 , the mobile phone may display a floating UI element 904 on the mobile phone screen, and the floating UI element 904 includes a link icon. The floating UI element 904 may be the first floating element in the embodiment of the present application. The link icon is used to link to the web page corresponding to the URL in the URL bar 902, that is, the URL copied by the user is displayed on the mobile phone screen in the form of a floating UI element.
[0156] It should be noted that the specific implementation process of displaying the URL on the mobile phone screen as a floating UI element is different from Figure 6 The specific implementation process of presenting the text element as a floating UI element on the mobile phone screen is similar, and will not be repeated here. The difference is that the drag event includes the URL address and the element type presented as a floating UI element is URL, and the element control module of the mobile application layer can use the link icon as a shadow.
[0157] Use the above Fig. 9The technical solution of the illustrated embodiment, for a URL address, the user can display the URL address on the mobile phone screen in the form of a floating UI element by performing a drag operation or a copy operation, such as adsorption display on the edge of the terminal screen. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view or share information across applications. For example, after the mobile phone receives the user's click operation on the floating UI element, the mobile phone can jump to the web page of the URL address presented by the floating UI element. For example, after the mobile phone displays the URL address in the form of a floating UI element on its screen, the mobile phone (such as the frame layer of the mobile phone) can associate the identifier of the floating UI element with the URL address and store it in the mobile phone. After the user clicks on the floating UI element, the mobile phone can open the URL address corresponding to the identifier of the floating UI element with the corresponding application according to the stored association relationship and display it to the user. The application can be a default application set by the user, or an application automatically selected by the system. And the floating UI element can be moved. For another example, when the user wants to use the URL across pages or applications, such as sharing the URL with a friend, by dragging the floating UI element to the input box of the chat interface with the friend, the URL in the floating UI element can be pasted into the input box to share with the friend, etc. The specific implementation is similar to the implementation of filling the text displayed in the form of a floating UI element into the input box, and will not be described in detail here.
[0158] Combination Fig.10 , an example is given to illustrate the process of displaying file resource elements on a mobile phone screen in the form of floating UI elements.
[0159] The file resource element may refer to pictures, documents, compressed packages and other types of file resources saved in the form of files in the mobile phone system. The operation that triggers the file resource element to be displayed in the form of a floating UI element may be a user's dragging operation on the file resource element.
[0160] For example, Fig.10 As shown in (a), the mobile phone displays the file resource management page 1001. The resource management page 1001 may be the first interface in the embodiment of the present application. When the user wants to display one or more file resources in the file resource management page 1001 on the mobile phone screen in the form of floating UI elements, the user may perform a drag operation on the file resource. After the user releases the drag, the mobile phone may display the file resource on the mobile phone screen in the form of a floating UI element.
[0161] The above drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the file resource to float and an operation of triggering the floating file resource to move.
[0162] For example, the operation that triggers the floating of a file resource is a long press and selection operation on the file resource, and the operation that triggers the movement of the floating file resource is a press and move operation on the floating file resource. Fig.10 As shown in (a) of FIG. 1 , the user can click the selection button 1002 in the file resource management page 1001. In response to this operation, Fig.10 As shown in (b) in the figure, the mobile phone enters the selection mode for the file resources in the file resource management page 1001. The user can select file resources in this selection mode. If the mobile phone receives the user's selection operation on the file resource "zyy" 1003, the file resource "zyy" 1003 is in a selected state. The file resource "zyy" 1003 can be the first element in the embodiment of the present application. The user can long press the selected file resource "zyy" 1003. In response to the user's operation, the mobile phone can display the file resource "zyy" 1003 as a floating state. For example, Fig.10 As shown in (c) in the figure, the mobile phone can perform background blur processing and display the floating file resource "zyy" 1003. Afterwards, the user presses the floating file resource "zyy" 1003 and moves the finger (such as moving the finger to the right), so that the file resource "zyy" 1003 moves on the mobile phone screen following the movement of the user's finger. Afterwards, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in Fig.10 As shown in (d) in the figure, the mobile phone can display a floating UI element 1004 on the mobile phone screen, and the floating UI element 1004 includes a link icon. The link icon is used to link to the storage address of the file resource "zyy" 1003 corresponding to the mobile phone, that is, the file resource dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. For example, the floating UI element 1004 can be adsorbed and displayed on the edge of the mobile phone screen (such as the left edge of the screen or the right edge of the screen, Fig.10 (d) in the figure is shown as an example of adsorption display on the right edge of the screen). The floating UI element 1004 may be the first floating element in the embodiment of the present application. The drag operation and the drag-and-release operation may be the second operation in the embodiment of the present application.
[0163] It should be noted that the specific implementation process of presenting file resource elements as floating UI elements on the mobile phone screen is different from Figure 6 The specific implementation process of presenting the text-type element as a floating UI element on the mobile phone screen is similar and will not be described here. The difference is that the drag event includes the storage address of the file resource in the mobile phone and the element type presented as a floating UI element is a file resource class, and the element control module of the mobile application layer can use the link icon as a shadow.
[0164] Similarly, using the above Fig.10 According to the technical solution of the illustrated embodiment, for file resource elements, the user can display the file resource elements on the mobile phone screen in the form of floating UI elements by performing a drag operation, such as being adsorbed and displayed on the edge of the terminal screen. And after receiving the user's operation of switching pages within the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view or share information across applications. For example, when the user wants to use the file resource across pages or across applications, such as sharing the file resource with a friend, the file resource corresponding to the floating UI element can be shared with the friend by dragging the floating UI element into the input box of the chat interface with the friend. The specific implementation is similar to the implementation of filling the input box with text displayed in the form of a floating UI element, and will not be repeated here.
[0165] Combination Fig.11 , which illustrates the process of displaying list elements on a mobile phone screen in the form of floating UI elements.
[0166] Among them, the list element is Android TM Native control types, such as ListView and RecyclerView, are all list elements. Specifically, they can be list elements in social, communication, shopping and other applications. The operation that triggers the display of list elements as floating UI elements can be the user's dragging operation on the elements included in the list items, such as the element representing the user's avatar in the list items. For example, Fig.11 As shown in (a) of FIG. 1 , the mobile phone currently displays the homepage 1101 of WeChat. The homepage 1101 of WeChat can be the first interface in the embodiment of the present application. When the user wants to open WeChat TM When a list element in the homepage 1101 of the mobile phone, such as a chat list item with one or more friends, is displayed on the mobile phone screen in the form of a floating UI element, a drag operation can be performed on the contact avatar in the list item. After the user releases the drag, the mobile phone can display the contact avatar on the mobile phone screen in the form of a floating UI element.
[0167] The drag operation may include one or more operations. For example, the drag operation may include an operation of triggering the contact avatar to float and an operation of triggering the floating contact avatar to move.
[0168] For example, the operation of triggering the floating of the contact avatar is a long press operation on the contact avatar, and the operation of triggering the movement of the floating contact avatar is an operation of pressing and moving the floating contact avatar. Fig.11 As shown in (a) in the figure, users can long press WeChat TMThe contact avatar 1102 of friend BB in the homepage 1101 of the mobile phone can be the first element in the embodiment of the present application. The mobile phone can receive a long press operation of the user on the contact avatar 1102. In response to the long press operation, the mobile phone can display the contact avatar 1102 in a floating state. Fig.11 As shown in (b) of the figure, the mobile phone can perform background blur processing and display a floating contact avatar 1102. Afterwards, the user presses the floating contact avatar 1102 and moves the finger (such as moving the finger to the right), so that the contact avatar 1102 moves on the mobile phone screen following the movement of the user's finger. Afterwards, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in Fig.11 As shown in (c), the mobile phone can display a floating UI element 1103 on the mobile phone screen, and the floating UI element 1103 includes a contact avatar 1102, that is, the contact avatar in the list item dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. The floating UI element 1103 can be the first floating element in the embodiment of the present application. The drag operation and the drag release operation can be the second operation in the embodiment of the present application.
[0169] It should be noted that the above Fig.11 The embodiment shown is described by taking the list element as a list item in WeChat as an example. The list element may also be a list item in other application interfaces, such as a list item in an address book, or a list item in a short message, and this embodiment does not make any specific limitation here.
[0170] Combination Figure 4 ,like Fig.12As shown, as an example, the specific implementation process of presenting the list-type elements as floating UI elements on the mobile phone screen may be: after the mobile phone, such as the application of the mobile phone application layer, receives the user's drag operation on an element in the list item, it may trigger the dragging of the element. The application of the mobile phone application layer may obtain the drag event and send it to the drag service module of the mobile phone application layer. For example, the process of the application of the mobile phone application layer obtaining the drag event may be: after the user performs the drag operation, the framework layer of the mobile phone may monitor the drag operation (such as a long press operation on the floating contact avatar), and in response, the framework layer of the mobile phone may generate a drag event (DragEvent). The application of the mobile phone application layer may obtain the drag event generated by the framework layer through callback. Among them, the drag event may include the coordinates of the pressing point when the user performs the long press operation and the type of the dragged element (such as the list class). After receiving the drag event, the drag service module of the mobile phone application layer may cooperate with the element control module of the mobile phone application layer to construct a shadow according to the coordinates of the pressing point in the drag event and the type of the dragged element (such as the list class). If the drag service module of the mobile application layer determines that the type of the dragged element is a list class, the drag service module of the mobile application layer can transmit the coordinates of the press point in the drag event to the window manager of the mobile frame layer. The window manager can determine the control within the preset range of the press point based on the coordinates of the press point, and take a screenshot of the control and transmit it to the element control module of the mobile application layer, so that the element control module generates a shadow based on the screenshot. Among them, the preset range of the press point can be within a predetermined pixel (such as 1 pixel, or greater than 1 pixel) distance centered on the press point. Afterwards, the element control module of the mobile application layer can transmit the generated shadow to the window manager of the mobile frame layer, so that the window manager displays the shadow in the form of a floating UI element. For example, the window manager displays a circular element on the mobile phone screen, and the circular element is a floating UI element, which includes a screenshot of the control at the user's press point, such as a contact avatar.
[0171] Use the above Fig.11The technical solution of the illustrated embodiment is that for list-type interface elements, the user can display the list items, such as the elements included in the list items, on the mobile phone screen in the form of floating UI elements by performing a drag operation. That is, the elements in the list items dragged by the user can be presented on the mobile phone screen through the floating UI element. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen. The floating UI element supports the operations supported by the original list item after being clicked. That is, the user performs a corresponding operation (the operation may be the fifth operation in the embodiment of the present application), such as the response made by the mobile phone after clicking the floating UI element (or the function executed, which may be the first function in the embodiment of the present application), which is the same as the response made by the mobile phone after the user clicks the original list item. The specific implementation may be: after the user performs a drag operation on an element in the list item, such as a contact avatar, the application of the mobile application layer may send the intent that needs to be triggered after clicking the contact avatar to the drag service module of the mobile application layer, so that after the mobile phone displays the contact avatar on its screen in the form of a floating UI element, the mobile phone may associate the identifier of the floating UI element with the above-mentioned intent that needs to be triggered and store it in the mobile phone. Afterwards, after the mobile phone receives the user's operation on the floating UI element, such as a click operation, the mobile phone may call the corresponding intent that needs to be triggered to make a corresponding response.
[0172] For example, combined with Fig.11 In the example shown, the user drags the contact avatar of his friend BB to trigger its display as a floating UI element on the phone screen, and then the user exits WeChat on the phone. TM Application, and as Fig.13A As shown in (a) of FIG. 1 , the mobile phone displays the desktop 1301. The contact avatar of friend BB continues to be displayed on the mobile phone screen in the form of a floating UI element, such as Fig.13A In addition, the mobile phone associates the identifier of the floating UI element 1302 with the intent to be triggered and stores it in the mobile phone. For example, the intent to be triggered is to display a chat interface with a friend BB. After the mobile phone receives the user's operation on the floating UI element 1302, such as a click operation, as a response to the operation, Fig.13A As shown in (b) in FIG. 1 , the mobile phone displays a chat interface 1303 with a friend BB.
[0173] For example, combining Fig.14 ,like Fig.14 As shown in (a) of FIG. 1 , the mobile phone displays the address book interface 1401. After receiving the user's drag operation on the avatar 1402 of the contact Sam in the address book interface 1401, the mobile phone displays the address book interface 1401. Fig.14As shown in (b), the mobile phone displays a floating UI element 1403, which includes the avatar 1402 of the contact Sam, that is, the avatar 1402 of the contact Sam in the address book interface 1401 dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. In addition, the mobile phone associates the identifier of the floating UI element 1403 with the intent to be triggered and stores it in the mobile phone. For example, the intent to be triggered is to make a call to the corresponding contact. Fig.14 As shown in (c) of FIG. 1 , the user triggers the mobile phone to exit the address book, and the mobile phone displays the desktop 1404. The floating UI element 1403 continues to be displayed on the mobile phone screen. Afterwards, after the mobile phone receives the user's operation on the floating UI element 1403, such as a click operation, in response to the operation, the mobile phone can call the contact Sam. Fig.14 As shown in (d) in FIG. 1 , the mobile phone may also display a dialing interface 1405 .
[0174] In addition, for contact avatars displayed in the form of floating UI elements, when content such as pictures, URLs, or text is dragged onto the floating UI element, the mobile phone can directly send the content to the corresponding contact. The operation of dragging content can include an operation to trigger the floating of content and an operation to trigger the movement of the floating content. For example, the operation to trigger the floating of content is a long press operation on the content, and the operation to trigger the movement of the floating content is an operation to press and move the floating content. Fig.11 In the example shown, the user drags the contact avatar of his friend BB, triggering it to be displayed on the phone screen as a floating UI element, and then opens the gallery application. Fig. 13B As shown in (a) of FIG. 1 , the mobile phone displays the interface 1304 of the gallery application. The contact avatar of friend BB continues to be displayed on the mobile phone screen in the form of a floating UI element, such as Fig. 13B After the mobile phone receives the user's long press operation on the picture 1306 in the interface 1304 of the gallery application, as shown in (a) of the floating UI element 1305. Fig. 13B As shown in (b) of FIG. 1 , the mobile phone displays the picture 1306 floating. Afterwards, the mobile phone receives the user's operation of pressing and moving the floating picture 1306, and the mobile phone displays the animation of the picture 1306 moving. If the mobile phone receives the user's operation of releasing the drag when moving the picture 1306 to the position of the floating UI element 1305, the mobile phone sends the picture 1306 to the friend BB. In addition, Fig. 13BAs shown in (c), the mobile phone can also automatically open the chat interface 1307 with the friend BB and display a message of successful sending. Among them, the specific implementation of the mobile phone sending pictures to friends can be that the mobile phone associates the identifier of the floating UI element with the intent to be triggered (for example, the intent to be triggered is to display the chat interface with the friend BB) and stores it in the mobile phone. After receiving the user's dragging of content (such as picture 1306) to the position of the floating UI element and releasing the drag operation, the mobile phone can call the corresponding intent to be triggered to start the chat interface. The mobile phone can also construct material data (ClipData) and save the content dragged by the user into the ClipData. Afterwards, the mobile phone releases the ClipData in the started chat interface, so that the application will process the ClipData, that is, send the content in the ClipData to the corresponding contact friend. It should be noted that Fig. 13B The example shown is to illustrate that dragging a picture directly onto a contact avatar presented as a floating UI element can trigger the mobile phone to send the picture directly to the corresponding contact. It can be understood that in this embodiment, when the user drags text, pictures, URLs, files, etc. onto a floating UI element that can respond to a send event (such as the contact avatar presented as a floating UI element above) and releases it, the send operation is triggered, which can improve the interaction efficiency of conventional operations such as sharing and sending content.
[0175] The above example is explained by taking the mobile phone associating the identifier of the floating UI element with the intent to be triggered and storing it in the mobile phone, which is used to trigger the mobile phone to make a corresponding response after receiving the user's operation on the floating UI element. In some other embodiments, the mobile phone may not associate the identifier of the floating UI element with the intent to be triggered and store it in the mobile phone, but after receiving the user's operation on the floating UI element, send the corresponding operation event to the application to trigger the application to make a corresponding response according to the operation event. For example, after receiving the user's operation on the floating UI element, the mobile phone can determine whether the intent corresponding to the identifier of the floating UI element is stored. If the intent corresponding to the identifier of the floating UI element is stored, the mobile phone can start the intent to respond to the user's operation. If it is not stored and the application is still displayed in the foreground, the mobile phone can transmit the corresponding operation event to the main screen to trigger the application to make a corresponding response according to the operation event. If the application retreats to the background, the mobile phone can move the interface of the application to the virtual screen for drawing. After determining that the intent corresponding to the identifier of the floating UI element is not stored, the mobile phone can send the corresponding operation event to the virtual screen drawing the application interface to trigger the application to make a corresponding response according to the operation event.
[0176] Combination Fig.15, an example is given to illustrate the process of displaying icon elements on a mobile phone screen in the form of floating UI elements.
[0177] The icon element may refer to a graphic element in the interface that has a click event defined therein. The operation that triggers the icon element to be displayed in the form of a floating UI element may be a user's dragging operation on the icon element.
[0178] For example, Fig.15 As shown in (a), the mobile phone displays Taobao TM Interface 1501. Taobao TM The interface 1501 may be the first interface in the embodiment of the present application. The interface 1501 includes multiple controls. When the user wants to display one or more controls in the Taobao interface 1501 on the mobile phone screen in the form of a floating UI element, the user may perform a drag operation on the control. After the user releases the drag, the mobile phone may display the control on the mobile phone screen in the form of a floating UI element. Among them, the above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation that triggers the control to float and an operation that triggers the floating control to move.
[0179] For example, the operation that triggers the floating of a control is a long press operation on the control, and the operation that triggers the movement of the floating control is an operation that presses and moves the floating control. Fig.15 As shown in (a) in the figure, users can TM The waiting-for-receiving control 1502 in the interface 1501 is long-pressed. The waiting-for-receiving control 1502 may be the first element in the embodiment of the present application. In response to the long-pressing operation, the mobile phone may display the waiting-for-receiving control 1502 as a floating state. Fig.15 As shown in (b) of the figure, the mobile phone can perform background blur processing and display a floating waiting-to-receive widget 1502. The user can press the floating waiting-to-receive widget 1502 and move the finger (such as moving the finger to the right) to make the waiting-to-receive widget 1502 move on the mobile phone screen following the movement of the user's finger. After that, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in Fig.15 As shown in (c) of FIG. 1 , the mobile phone may display a floating UI element 1503 on the mobile phone screen, and the floating UI element 1503 includes an icon corresponding to the to-be-received control 1502, that is, the control dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. The floating UI element 1503 may be the first floating element in the embodiment of the present application. The drag operation and the drag-release operation may be the second operation in the embodiment of the present application.
[0180] It should be noted that the specific implementation process of displaying icon elements as floating UI elements on the mobile phone screen is different from Fig.12 The specific implementation process of presenting the list-like elements as floating UI elements on the mobile phone screen is similar, and will not be repeated here. The difference is that the mobile phone receives the user's drag operation on the control, such as a long press operation, and can trigger the drag. The type of the dragged element in the drag event is the icon class.
[0181] It is understandable that the above Fig.15 The technical solution of the illustrated embodiment, for icon-like elements, the user can display the icon-like elements in the form of floating UI elements on the mobile phone screen by performing a drag operation. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen. The floating UI element supports the operations supported after the control is clicked. That is, the user performs a corresponding operation (the operation can be the fifth operation in the embodiment of the present application), such as the response made by the mobile phone after clicking the floating UI element (or the executed function, which can be the first function in the embodiment of the present application), which is the same as the response made by the mobile phone when the user clicks the original control. Its specific implementation can be: if the application is still displayed in the foreground, after receiving the user's operation on the floating UI element, the mobile phone can transmit the corresponding operation event to the main screen to trigger the application to make a corresponding response according to the operation event. If the application retreats to the background, the mobile phone can move the application to the virtual screen for drawing. After receiving the user's operation on the floating UI element, the mobile phone can send the corresponding operation event to the virtual screen that draws the application interface to trigger the application to make a corresponding response according to the operation event. In other words, combined with Fig.15 In the example shown, after receiving an operation from the user on the floating UI element 1503, such as a click operation, the mobile phone may display a detailed interface of the goods to be received.
[0182] Combination Fig.16 and Fig.17 , which gives an example of how card-type elements are displayed on a mobile phone screen in the form of floating UI elements.
[0183] Among them, the card-type element may refer to an element used to display information. Which elements in the interface are card-type elements may be pre-defined. For example, a corresponding configuration file may be pre-stored in the mobile phone, and the configuration file includes an identifier of the card-type element. For example, it may be an element in a navigation or shopping application. In some embodiments, the operation of triggering the display of the card-type element in the form of a floating UI element may be a user's dragging operation on the card-type element.
[0184] For example, Fig.16As shown in (a), the mobile phone displays a navigation interface 1601. The navigation interface 1601 may be the first interface in the embodiment of the present application. When the user wants to display one or more elements in the navigation interface 1601 on the mobile phone screen as floating UI elements, the user may perform a drag operation on the element. After the user releases the drag, the mobile phone may display the element on the mobile phone screen as a floating UI element. The above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation to trigger the element to float and an operation to trigger the floating element to move.
[0185] For example, the operation that triggers the element to float is a long press operation on the display area where the element is located, and the operation that triggers the floating element to move is an operation that presses and moves the floating element. Fig.16 As shown in (a) of FIG. 1 , the user can perform a long press operation on the display area where element 1602 in navigation interface 1601 is located. Element 1602 may be the first element in the embodiment of the present application. In response to the long press operation, the mobile phone may extract some card-like elements in navigation interface 1601 according to the configuration file, such as extracting card-like elements within a preset range of the pressing point and displaying the extracted card-like elements as floating. Fig.16 As shown in (a) of FIG. 1 , the mobile phone extracts element 1602 and element 1603 in navigation interface 1601. Element 1603 may be the second element in the embodiment of the present application. Fig.16 As shown in (b) of FIG. 1 , the mobile phone performs background blur processing and displays a floating element 1604, which includes the extracted element 1602 and the element 1603. The user presses the floating element 1604 and moves the finger (such as moving the finger to the right), so that the element 1604 moves on the mobile phone screen following the movement of the user's finger. After that, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in FIG. Fig.16As shown in (c), the mobile phone can display a floating UI element 1605 on the mobile phone screen, and the floating UI element 1605 includes the extracted element 1602 and the element 1603, that is, the element dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. Floating UI element 1605 can be the first floating element in the embodiment of the present application. Among them, when the extracted element is displayed in the form of a floating UI element, its layout can be the same as or different from the layout of the element in the interface. In the case where the layout of the extracted element when displayed in the form of a floating UI element is the same as its layout in the interface, after the mobile phone extracts the element, it can layout the extracted element according to its layout in the current interface, and then display it on the mobile phone screen in the form of a floating UI element. In the case where the layout of the extracted element when displayed in the form of a floating UI element is different from its layout in the interface, as an example, a corresponding layout file can be pre-stored in the mobile phone, and the layout file includes the layout configuration of the extracted element, such as the layout configuration including element 1602 and element 1603. After extracting elements, such as the above-mentioned element 1602 and element 1603, the mobile phone can lay out the element 1602 and element 1603 according to the layout file and display them in the form of floating UI elements, such as the display effect as shown in FIG. Fig.16 The floating UI element 1605 shown in (c) in FIG. 1 may be a second operation in an embodiment of the present application.
[0186] It should be noted that Fig.16 In the example shown, the specific implementation process of presenting card-like elements as floating UI elements on the mobile phone screen is similar to Fig.12 The specific implementation process of presenting the list-type elements as floating UI elements on the mobile phone screen is similar, and will not be repeated here one by one. The difference is that the mobile phone receives the user's drag operation on the element, such as receiving a long press operation on the display area where the element is located, and the drag can be triggered. The type of the dragged element in the drag event is a card class. In addition, in the specific implementation, the mobile phone can use the card elements within the preset range of the press point as the elements to be subsequently presented as floating UI elements according to the coordinates of the user's press point.
[0187] In some other embodiments, the operation of triggering the display of a card-type element in the form of a floating UI element may be an operation of a user on a predetermined button in the interface.
[0188] For example, Fig.17As shown in (a) of FIG. 1 , the mobile phone displays a shopping details interface 1701 of a shopping application. When the user wants to display one or more elements in the shopping details interface 1701 on the mobile phone screen in the form of floating UI elements, the user can operate (e.g., click) a predetermined button, such as the share button 1706. After receiving the user's click operation on the share button, the mobile phone can display some elements in the shopping details interface 1701, such as card-type elements, on the mobile phone screen in the form of floating UI elements.
[0189] For example, continue to refer to Fig.17 As shown in (a) of FIG. 1 , the user operates, for example, clicks, the share button 1706 in the shopping details interface 1701 of the shopping application. In response to the click operation, the mobile phone can extract card-like elements in the shopping details interface 1701, such as the product preview 1702, the product price 1707, the product introduction 1703, the add to cart button 1704, and the buy now button 1705. Fig.17 As shown in (b) in FIG. 1 , the mobile phone may display a floating UI element 1708 on the mobile phone screen, wherein the floating UI element 1708 includes an extracted product preview 1702 , a product price 1707 , a product introduction 1703 , an add to cart button 1704 and a buy now button 1705 .
[0190] After receiving the user's operation on a predetermined button in the interface, the mobile phone can extract the card-type elements in the interface according to the corresponding configuration file so that they can be presented to the user in the form of floating UI elements. The configuration file defines which elements in the interface are presented in the form of floating UI elements after receiving the user's operation on the predetermined button.
[0191] It is understandable that the above Fig.16 and Fig.17 The technical solution of the embodiment shown is that for card-like elements, the user can display the card-like elements on the mobile phone screen in the form of floating UI elements by performing a drag operation or operating a predetermined button. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, which is convenient for the user to continue viewing. In addition, it can also make it more convenient to share information across applications. For example, when the user wants to share the card-like element with a friend, by placing the floating UI element (such as Fig.17By dragging the floating UI element 1708 shown in (b) of FIG. 1708 ) into the input box of the chat interface with the friend, the floating UI element can be shared with the friend. The specific implementation is similar to the implementation of filling the text displayed in the form of a floating UI element into the input box and sharing it with the friend, which will not be described in detail here. In addition, after the friend receives the floating UI element, he can click the received floating UI element in the chat interface displayed on the mobile phone to trigger the mobile phone to display the corresponding shopping details interface.
[0192] Combination Fig.18 , an example is given to illustrate the process of displaying a video window element on a mobile phone screen in the form of a floating UI element.
[0193] Among them, the video window element is Android TM Native control types, such as TextureView, SurfaceView, or the video element in the webview are all video window elements. The operation that triggers the video window element to be displayed as a floating UI element can be a user's drag operation on the video window element.
[0194] For example, Fig.18 As shown in (a) in the figure, the mobile phone displays the video details interface 1801. The video details interface 1801 may be the first interface in the embodiment of the present application. When the user wants to display the video element in the video details interface 1801 on the mobile phone screen as a floating UI element, the user may perform a drag operation on the video element. After the user releases the drag, the mobile phone may display the video element on the mobile phone screen as a floating UI element. The above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation to trigger the floating of the video element and an operation to trigger the movement of the floating video element.
[0195] For example, the operation of triggering the floating of the video element is a long press operation on the video element, and the operation of triggering the movement of the floating video element is an operation of pressing and moving the floating video element. Fig.18 As shown in (a) of FIG. 1 , the user can long-press the video element 1802 in the video details interface 1801. The video element 1802 can be the first element in the embodiment of the present application. In response to the long-press operation, the mobile phone can display the video element 1802 in a floating state. Fig.18As shown in (b) of FIG. 1 , the mobile phone can perform background blur processing and display a floating video element 1802. The user can press the floating video element 1802 and move the finger (such as moving the finger to the right) to make the video element 1802 move on the mobile phone screen following the movement of the user's finger. After that, the user can release the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag release operation. After the mobile phone receives the drag release operation, as shown in FIG. Fig.18 As shown in (c) of FIG. 1 , the mobile phone may display a floating UI element 1803 on the mobile phone screen, and the floating UI element 1803 includes a video element 1802, that is, the video element dragged by the user is displayed on the mobile phone screen in the form of a floating UI element. The floating UI element 1803 may be the first floating element in the embodiment of the present application. The drag operation and the drag release operation may be the second operation in the embodiment of the present application.
[0196] Among them, the specific implementation process of presenting the video window class element as a floating UI element on the mobile phone screen can be: after the mobile phone, such as the application of the mobile phone application layer receives the user's drag operation on the video window class element, it can trigger the drag of the video window class element. The application of the mobile phone application layer can obtain the drag event and send it to the drag service module of the mobile phone application layer. For example, the process of the application of the mobile phone application layer obtaining the drag event can be: after the user performs the drag operation, the framework layer of the mobile phone can monitor the drag operation (such as a long press operation on the video window class element), and in response, the framework layer of the mobile phone can generate a drag event (DragEvent). The application of the mobile phone application layer can obtain the drag event generated by the framework layer through callback. Among them, the drag event may include the coordinates of the pressing point, the identifier of the application (such as the package name), the identifier of the video window class element and the type of the dragged element (such as the video window class). The drag service module of the mobile phone application layer can start the virtual screen and pass the identifier of the virtual screen and the package name of the application to the window manager of the mobile phone framework layer. The window manager of the mobile phone framework layer can move the interface of the application indicated by the package name, that is, the above-mentioned video details interface, to the virtual screen for drawing according to the identifier of the virtual screen and the package name of the application. The drag service module of the mobile phone application layer can also cooperate with the element control module of the mobile phone application layer to construct a shadow according to the drag event. If the drag service module of the mobile phone application layer determines that the type of the dragged element is a video window class, the drag service module of the mobile phone application layer can transmit the coordinates of the pressing point in the drag event to the window manager of the mobile phone framework layer. The window manager can determine the element near the pressing point based on the coordinates of the pressing point, and take a screenshot of the element and transmit it to the element control module of the mobile phone application layer, so that the element control module generates a shadow based on the screenshot, and transmits the generated shadow to the window manager of the mobile phone framework layer, so that the window manager adds the shadow to the main screen of the mobile phone. Afterwards, when the user releases the drag, the drag service module of the mobile application layer can extract the corresponding video window class elements, such as video elements, from the interface drawn on the virtual screen according to the identifier of the video window class elements in the drag event, and map the extracted video window class elements to the shadow of the main screen, thereby achieving the effect of displaying the video window class elements in the form of floating UI elements.
[0197] Use the above Fig.18 In the technical solution of the embodiment shown, for the video window element, the user can display the video window element as a floating UI element on the mobile phone screen by performing a drag operation. And when receiving a user switching pages in the application, or exiting the application (such as Fig.18 After displaying the desktop 1804 as shown in (d) in the figure, or opening other applications, the floating UI element (such as Fig.18The floating UI element 1803 shown in (d) in the figure can continue to be displayed on the mobile phone screen for the user to continue viewing. This makes it convenient for the user to operate other interfaces of the mobile phone while continuing to view the video displayed by the video window element. In addition, when the video carried by the video window element is a local video, the user can Fig.18 The video can be shared with friends by dragging the floating UI element 1803 shown in (c) of FIG. 1 into the input box of the chat interface with the friend. The specific implementation is similar to the implementation of filling the text displayed in the form of a floating UI element into the input box and sharing it with friends, which will not be described in detail here.
[0198] It is understandable that an interface may include multiple interface elements. The interface elements that support display in the form of floating UI elements in the interface may be customized by a third-party application or recommended by the system. That is to say, a third-party application or a system may define that one or more interface elements in the interface support presentation as floating UI elements. For example, the developer of the third-party application may predefine which elements in the interface of the application support presentation as floating UI elements, and configure the corresponding configuration file in the mobile phone. For another example, the system developer may configure the corresponding recommendation model in the mobile phone, which is used to identify which elements in the interface support presentation as floating UI elements. Based on this, in order to facilitate the user to know which elements in the interface support presentation as floating UI elements, in this embodiment, when the mobile phone displays the interface, after receiving the corresponding user operation, the elements that support presentation as floating UI elements in the interface (the interface may be the first interface in the embodiment of the present application) are displayed in the first form, such as highlighting, and the elements that do not support presentation as floating UI elements are displayed in the second form. Among them, the user operation may be a preset operation of the user on the elements in the interface, such as a long press operation, a double-click operation, etc. The user operation may also be an operation of the user on the corresponding switch button. The user operation may be the first operation in the embodiment of the present application. The switch button may be a predetermined function control in the embodiment of the present application.
[0199] For example, the user operation is a long press operation of an element in the interface. Fig.19 , display Taobao on mobile phone TM For example, interface 1901 of Fig.19 As shown in (a) in the figure, the user can long-press the pending payment control 1902 in the interface 1901. After the mobile phone receives the long-press operation on the pending payment control 1902, it can determine whether the pending payment control 1902 supports presentation as a floating UI element. If the pending payment control 1902 supports presentation as a floating UI element, the mobile phone can identify all elements in the interface 1901 that support presentation as floating UI elements according to the corresponding configuration file or recommendation model. Fig.19 As shown in (b) of FIG. 1 , the mobile phone recognizes that the pending payment control 1902, the pending shipment control 1903, the pending receipt control 1904, the evaluation control 1905, the my express control 1906 and the collection center control 1907 support being presented as floating UI elements. Afterwards, the mobile phone can highlight these elements that support being presented as floating UI elements, such as displaying them in a floating state. Fig.19 As shown in (b) in the figure, the mobile phone may perform background blur processing and display floating pending payment control 1902, pending shipment control 1903, pending receipt control 1904, evaluation control 1905, my express control 1906 and collection center control 1907. If the pending payment control 1902 does not support presentation as a floating UI element, the mobile phone may not respond.
[0200] For another example, the user operation is an operation of a corresponding switch button by the user. Fig. 20 , continue to display Taobao on your mobile phone TM Take the interface 2001 as an example. Fig. 20 As shown in (a) of FIG. 2 , when the user wants to check which elements in the interface 2001 support presentation as floating UI elements, the user can perform a sliding operation from the upper edge of the mobile phone screen to the lower edge. In response to the sliding operation, as shown in FIG. Fig. 20 As shown in (b) of FIG. 1 , the mobile phone displays a menu bar 2002, and the menu bar 2002 includes a switch button 2003 for the floating UI element function. The user can operate the switch button 2003, such as clicking. After the mobile phone receives the user's click operation on the switch button 2003, the mobile phone can identify all elements in the interface 2001 that support presentation as floating UI elements according to the corresponding configuration file or recommendation model, and display the elements that support presentation as floating UI elements as floating. For example, Fig. 20 As shown in (c) in the figure, the mobile phone can perform background blur processing and support the elements presented in the form of floating UI elements: pending payment control 2004, pending shipment control 2005, pending receipt control 2006, evaluation control 2007, my express control 2008 and collection center control 2009 are displayed in a floating manner.
[0201] By highlighting the elements in the interface that support presentation as floating UI elements and blurring other elements, the user is prompted which elements in the interface support presentation as floating UI elements. It should be noted that in addition to displaying the elements that support presentation as floating UI elements as floating, they can also be enlarged for display, or only the elements that support presentation as floating UI elements can be enlarged for display. This embodiment does not specifically limit the form of highlighting. For example, Fig.21 As shown, continue to display Taobao on your mobile phone TMAfter the mobile phone receives the user's preset operation on the element in the interface or the operation on the corresponding switch button, in response, the mobile phone can add a dark mask to the current interface, such as Fig.21 In 2101, elements that support presentation as floating UI elements are placed on the dark mask (e.g., pending payment control 2102, pending shipment control 2103, pending receipt control 2104, evaluation control 2105, my express control 2106, and collection center control 2107 are placed on the dark mask), and other elements are placed under the dark mask. Of course, elements that support presentation as floating UI elements can also be enlarged. Afterwards, the user can trigger the corresponding element to be displayed on the mobile phone screen in the form of a floating UI element by performing a corresponding operation (e.g., dragging and dropping) or turning off the corresponding switch button, the interface can restore to its initial display state.
[0202] According to the description of the above embodiments, the user can trigger the elements in the interface to be presented on the mobile phone screen in the form of floating UI elements by performing corresponding operations. In some embodiments, only one floating UI element may be allowed to be displayed on the mobile phone screen. In some other embodiments, multiple floating UI elements may be allowed to be displayed simultaneously on the mobile phone screen. When multiple floating UI elements are allowed to be displayed simultaneously, the maximum number of floating UI elements allowed to be displayed simultaneously (such as the maximum number is N) may be limited. For example, the system default configuration may allow multiple floating UI elements to be displayed on the mobile phone screen at the same time, such as the system default configuration allows only one floating UI element to be displayed on the mobile phone screen. For another example, the mobile phone may provide a configuration interface for the user to choose whether to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time, such as the user chooses to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time. In the case where the user chooses to allow multiple floating UI elements to be displayed simultaneously, the user may also set the maximum number of floating UI elements allowed to be displayed simultaneously. For another example, the user may choose whether to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time. In the case where the user chooses to allow multiple floating UI elements to be displayed simultaneously, the maximum number of floating UI elements allowed to be displayed simultaneously is the default configuration of the system.
[0203] In the case where multiple floating UI elements are allowed to be displayed on the mobile phone screen at the same time, the user can trigger the mobile phone to display multiple elements on the mobile phone screen in the form of floating UI elements by performing corresponding operations (such as dragging operations, copying operations, etc.). In some embodiments, the mobile phone can arrange multiple floating UI elements according to preset arrangement rules. For example, the mobile phone arranges multiple floating UI elements on the mobile phone screen in sequence according to the order in which the floating UI elements are generated, such as arranging them from top to bottom on the edge of the mobile phone screen. For example, in combination with Fig. 22 ,like Fig. 22As shown in (a) of FIG. 1 , the mobile phone displays a floating UI element 2201 on the left edge of the mobile phone screen. The user triggers the mobile phone to generate a floating UI element 2202. Fig. 22 As shown in (b) of FIG. 1 , the mobile phone can arrange and display the floating UI element 2202 behind the floating UI element 2201. The user triggers the mobile phone to generate the floating UI element 2203, as shown in FIG. Fig. 22 As shown in (c) in FIG. 1 , the mobile phone may arrange and display the floating UI element 2203 after the floating UI element 2201 and the floating UI element 2202. That is, the mobile phone may display the earliest generated floating UI element at the top, and the subsequently generated floating UI elements may be arranged in the order of the time of generation.
[0204] In some embodiments, when multiple floating UI elements are allowed to be displayed simultaneously on the mobile phone screen and the maximum number of floating UI elements allowed to be displayed simultaneously is set, if the number of floating UI elements already displayed on the mobile phone screen has reached the maximum limit, when the user triggers the mobile phone to display interface elements in the form of floating UI elements again, the mobile phone can automatically delete the earliest generated floating UI element and display the floating UI element generated this time on the mobile phone screen. Fig.23 , the phone currently displays 5 floating UI elements, and the maximum number of floating UI elements allowed to be displayed simultaneously is 5. Fig.23 As shown in (a) of FIG. 1 , the mobile phone currently displays five floating UI elements, namely floating UI element 2302, floating UI element 2303, floating UI element 2304, floating UI element 2305 and floating UI element 2306. Among them, floating UI element 2302 is the earliest generated floating UI element. Fig.23 As shown in (a) of the figure, the mobile phone displays the interface 2301 of the gallery application, which may be the second interface in the embodiment of the present application. The user drags the picture 2307 in the interface 2301 to trigger the picture 2307 to be displayed on the mobile phone screen as a floating UI element. The picture 2307 may be the third element in the embodiment of the present application. Since the number of floating UI elements displayed on the mobile phone screen has reached the upper limit at this time, the mobile phone receives the user's drag and release operation, such as Fig.23As shown in (b) in the figure, the mobile phone can delete (or not display) floating UI element 2302, and display floating UI element 2308, as well as floating UI element 2303, floating UI element 2304, floating UI element 2305 and floating UI element 2306 on the screen. For example, the mobile phone can automatically arrange and display the newly generated floating UI element 2308 (floating UI element 2308 can be the second floating element in the embodiment of the present application) at the bottom of the displayed floating UI element queue, and the displayed floating UI elements move up in sequence, and the floating UI element 2302 at the top of the queue is automatically replaced and disappears. In some embodiments, before the newly generated floating UI element automatically replaces the earliest generated floating UI element and is displayed on the screen, the mobile phone can also display a prompt message to prompt the user that the newly generated floating UI element will replace the earliest generated floating UI element and be displayed on the screen. After the prompt, the newly generated floating UI element automatically replaces the earliest generated floating UI element and is displayed on the screen.
[0205] Among them, the above Fig.23 In the illustrated embodiment, when the number of floating UI elements already displayed on the mobile phone screen has reached the maximum number limit, when the user triggers the mobile phone to display the interface elements in the form of floating UI elements again, whether to allow the newly generated floating UI elements to automatically replace the earliest generated floating UI elements to be displayed on the screen can be the default configuration of the system, or a setting interface can be provided for the user to set. For example, when the user sets to allow the newly generated floating UI elements to automatically replace the earliest generated floating UI elements to be displayed on the screen, the number of floating UI elements already displayed on the mobile phone screen has reached the maximum number limit, and when the user triggers the mobile phone to display the interface elements in the form of floating UI elements again, the mobile phone can automatically replace the earliest generated floating UI elements with the newly generated floating UI elements to be displayed on the screen. If the user sets not to allow the newly generated floating UI elements to automatically replace the earliest generated floating UI elements to be displayed on the screen, the number of floating UI elements already displayed on the mobile phone screen has reached the maximum number limit, and when the user triggers the mobile phone to display the interface elements in the form of floating UI elements again, the mobile phone displays a prompt message, such as the prompt message can be "the number has reached the upper limit, please delete the existing floating UI elements", prompting the user that the number of floating UI elements on the screen has reached the maximum number limit, and no new floating UI elements can be added to display. Of course, the above Fig.23 The automatic replacement rule in the illustrated embodiment, such as automatically replacing the earliest generated floating UI element, is only an example. In some other embodiments, the automatic replacement rule may also be to automatically replace the latest generated floating UI element, or to automatically replace the least frequently used floating UI element, etc. This embodiment does not limit this. The automatic replacement rule may be predefined by the system, or a setting interface may be provided for the user to set.
[0206] As described in the above embodiments, the user can trigger the mobile phone to display various types of elements (such as text, picture, icon, card, list contact, video window, file resource) in the interface on the mobile phone screen in the form of floating UI elements by performing corresponding operations (such as dragging, copying, etc.). Therefore, it can be considered that the corresponding generated floating UI element also has a corresponding type, such as the type of the floating UI element is the same as the type of the element. Based on this, in some other embodiments, multiple floating UI elements are allowed to be displayed on the mobile phone screen at the same time, and the maximum number of floating UI elements allowed to be displayed at the same time is set. If the number of floating UI elements already displayed on the mobile phone screen has reached the maximum number limit, then when the user triggers the mobile phone to display interface elements in the form of floating UI elements again, the mobile phone can first determine whether there is a floating UI element of the same type as the newly generated floating UI element (or whether there is a floating UI element of the same type as the interface element) among the currently displayed floating UI elements. If not, it can be determined according to Fig.23 The embodiment shown displays floating UI elements on the mobile phone screen. If there are, such as floating UI element 1 in the currently displayed floating UI elements and the newly generated floating UI element of the same type, the newly generated floating UI element (the floating UI element may be the second floating element in the embodiment of the present application) and the floating UI element 1 (the floating UI element 1 may be the third floating element in the embodiment of the present application) may be automatically merged into a floating UI element group and displayed on the mobile phone screen.
[0207] For example, combined with Fig.24 ,like Fig.24 As shown in (a), the mobile phone currently displays 5 floating UI elements, namely floating UI element 2402, floating UI element 2403, floating UI element 2404, floating UI element 2405 and floating UI element 2406. Among them, floating UI element 2402 is a floating UI element of the list contact class. Floating UI element 2403 is a floating UI element of the file resource class. Floating UI element 2404 is a floating UI element of the icon class. Floating UI element 2405 is a floating UI element of the picture class. Floating UI element 2406 is a floating UI element of the card class. Fig.24 As shown in (a), the mobile phone displays the interface 2401 of the gallery application. The user drags the picture 2407 in the interface 2401 to trigger the picture 2407 to be displayed on the mobile phone screen in the form of a floating UI element (the type of the newly generated floating UI element is picture class). Because the number of floating UI elements displayed on the mobile phone screen has reached the upper limit at this time, after receiving the user's drag and release operation, the mobile phone can determine whether there is a floating UI element of the same type (picture class) as the newly generated floating UI element among the currently displayed floating UI elements. Fig.24 As can be seen from the example, the mobile phone can determine that the currently displayed floating UI element 2405 is of the same type as the newly generated floating UI element, both of which are picture types, and the mobile phone will merge the newly generated floating UI element with the floating UI element 2405 for display, such as Fig.24 As shown in (c) of FIG. 2 , the mobile phone displays a floating UI element group 2408. In some embodiments, when the mobile phone determines that there is a floating UI element of the same type as the newly generated floating UI element among the currently displayed floating UI elements, the mobile phone may also highlight the displayed floating UI element of the same type as the newly generated floating UI element to prompt the user that the newly generated floating UI element will be displayed in combination with the floating UI element. For example, Fig.24 As shown in (b) of FIG. 1 , when the user drags the image 2407, the mobile phone can perform background blur processing and display the floating image 2407. The mobile phone can also clearly display the floating UI element 2405 of the same type as the newly generated floating UI element among the displayed floating UI elements, and blur the other floating UI elements. Fig.24 As shown in (c) of FIG. 2 , the mobile phone displays a floating UI element group 2408. In addition, after the mobile phone combines and displays multiple (two or more) floating UI elements, the mobile phone may also display a prompt message on the floating UI element group to prompt the number of floating UI elements included in the floating UI element group. Fig.24 As shown in (c) in the figure, the mobile phone displays a superscript 2409 on the floating UI element group 2408 (such as the upper right corner), and the number "2" included in the superscript 2409 is used to indicate that the floating UI element group 2408 includes 2 floating UI elements, or is composed of 2 floating UI elements merged.
[0208] It should be noted that the above example is explained by taking the case where there is one floating UI element of the same type as the newly generated floating UI element among the displayed floating UI elements as an example. In some other embodiments, when there are multiple floating UI elements of the same type as the newly generated floating UI element among the displayed floating UI elements, the mobile phone may automatically select one or more of the multiple floating UI elements and merge them with the newly generated floating UI element into one floating UI element group. The mobile phone may also merge the floating UI element selected by the user among the multiple floating UI elements and the newly generated floating UI element into one floating UI element group according to the user's operation.
[0209] above Fig.24The illustrated embodiment is described by taking the case where the number of floating UI elements displayed on the screen reaches the upper limit and a new floating UI element is generated, and the mobile phone merges the floating UI elements of the same type into a floating UI element group and displays them on the mobile phone screen as an example. In some other embodiments, in the process of generating floating UI elements, if a user's operation triggers the mobile phone to generate multiple floating UI elements, the mobile phone can also automatically merge the floating UI elements of the same type among the generated multiple floating UI elements into a floating UI element group and present it to the user. In other words, when the user simultaneously triggers multiple elements of the same type to be presented in the form of floating UI elements, the mobile phone will automatically merge the floating UI elements of the generated multiple elements into a floating UI element group and present it to the user.
[0210] For example, combined with Fig.25 , take the case where a user triggers multiple image elements to be presented as floating UI elements at the same time, and the operation that triggers the presentation of the image elements as floating UI elements is a drag operation. For example, the drag operation includes an operation that triggers the image to float and an operation that triggers the floating image to move. The operation that triggers the image to float is a long press operation on the image, and the operation that triggers the floating image to move is an operation that presses and moves the floating image.
[0211] like Fig.25 As shown in (a) of FIG. 1 , the mobile phone displays an interface 2501 of the gallery application, and the interface 2501 of the gallery application includes multiple pictures. The user can long press the picture 2502 in the interface 2501 of the gallery application. Fig.25 As shown in (b) of FIG. 1 , the user long presses the picture 2503 in the interface 2501 of the gallery application. The mobile phone can receive the user's long press operation on the pictures 2502 and 2503. In response to the long press operation, the mobile phone can display the pictures 2502 and 2503 in a floating state. Fig.25 As shown in (c) in the figure, the mobile phone can perform background blur processing and display floating pictures 2502 and 2503. Afterwards, the user can press the floating pictures 2502 and 2503 and move his finger (e.g., move his finger to the right) to make pictures 2502 and 2503 move on the mobile phone screen following the movement of the user's finger. Afterwards, the user can release his finger to release the drag. After the user releases his finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the drag and release operation, the mobile phone can generate a floating UI element for picture 2502 and a floating UI element for picture 2503. Since these two floating UI elements are of the same type, both are picture types, as shown in Fig.25As shown in (d) of FIG. 1 , the mobile phone may display a floating UI element group 2504 on the mobile phone screen. The floating UI element group 2504 includes the floating UI element of the picture 2502 and the floating UI element of the picture 2503, or is a combination of the floating UI element of the picture 2502 and the floating UI element of the picture 2503. Fig.25 As shown in (d) in the figure, the mobile phone displays a superscript 2505 on the floating UI element group 2504 (such as the upper right corner), and the number "2" included in the superscript 2505 is used to indicate that the floating UI element group 2504 includes 2 floating UI elements, or is composed of 2 floating UI elements merged.
[0212] Similarly, when the user triggers multiple other elements of the same type at the same time, such as file resource class, text class, icon class, card class, list contact class, video window class, etc., to be presented in the form of floating UI elements, the mobile phone can also automatically merge the generated multiple floating UI elements into a floating UI element group and display them on the mobile phone screen. The mobile phone can also display a prompt message on the floating UI element group to prompt the number of floating UI elements included in the floating UI element group. In addition, in this embodiment, the number of floating UI elements allowed to be included in a floating UI element group can be unlimited, or an upper limit (e.g., 9) can be set. Among them, the upper limit can provide a setting interface for the user to set, or it can be the default configuration of the system.
[0213] For floating UI elements presented in a floating UI element group, the mobile phone can also expand the floating UI element group according to the user operation so that the user can view each floating UI element in the floating UI element group. Fig.26 As shown in (a), the mobile phone currently displays: floating UI element 2601, floating UI element 2602, floating UI element 2603, floating UI element group 2604 and floating UI element 2605. Among them, floating UI element group 2604 includes two floating UI elements. The mobile phone receives the user's operation on floating UI element group 2604 (the operation can be the third operation in the embodiment of the present application), such as a click operation, a double-click operation or a long press operation. In response to the operation, the mobile phone can display the floating UI elements included in the floating UI element group 2604. For example, Fig.26 As shown in (b) of FIG. 1 , the mobile phone hides other floating UI elements, namely, floating UI element 2601, floating UI element 2602, floating UI element 2603 and floating UI element 2605, expands floating UI element group 2604, and displays floating UI element 2604-1 and floating UI element 2604-2. Afterwards, upon receiving a user operation, such as an operation on an area other than the area where the floating UI elements are displayed, the mobile phone can automatically collapse the floating UI element group and redisplay the other floating UI elements, and the display effect is restored. Fig.26 As shown in (a) in the figure. In addition, as described in the above embodiment, there may be an upper limit requirement on the number of floating UI elements that are allowed to be displayed on the mobile phone screen at the same time. When the number of floating UI elements included in the floating UI element group is greater than the upper limit, after receiving the user's operation on the floating UI element group, the mobile phone may display some of the floating UI elements in the floating UI element group on the mobile phone screen, and other floating UI elements are temporarily not displayed. Among them, the number of floating UI elements displayed is the same as the upper limit. Afterwards, when the mobile phone receives the user's up and down sliding operation, it can trigger the mobile phone to switch the floating UI elements in the floating UI element group displayed.
[0214] For floating UI elements presented in a floating UI element group, the mobile phone can also respond to user operations (the operation is similar to Fig.26 The user operation in the embodiment shown is different), the floating UI element group is split (or separated) so that each floating UI element in the floating UI element group is displayed separately on the mobile phone screen. Fig. 27 As shown in (a) of FIG. 1 , the mobile phone currently displays a floating UI element group 2701. The floating UI element group 2701 includes two floating UI elements. The mobile phone receives an operation that the user can perform on the floating UI element group 2701, such as a click operation, a double-click operation, or a long press operation. In response to the operation, Fig. 27 As shown in (b), the mobile phone displays the floating UI element 2702 and the floating UI element 2703 included in the floating UI element group 2701 independently on the mobile phone screen. In addition, as described in the above embodiment, there may be an upper limit requirement for the number of floating UI elements allowed to be displayed on the mobile phone screen at the same time. After receiving the user's operation for separating the floating UI element group, the mobile phone may first determine whether all the floating UI element groups displayed on the screen exceed the upper limit after the floating UI element group is separated and displayed. If it does not exceed the upper limit, in response to the operation for separating the floating UI element group, the floating UI element group is separated and displayed on the mobile phone screen. If it exceeds the upper limit, the mobile phone may not perform the operation of separating the floating UI element group. The mobile phone may temporarily expand the floating UI element group for display, and then automatically retract it to continue to display it in the form of a floating UI element group.
[0215] In this embodiment, for the floating UI element displayed on the mobile phone screen, the mobile phone can delete it automatically or according to user operation (the user operation can be the fourth operation in the embodiment of the present application), and it will no longer be displayed on the mobile phone screen.
[0216] For example, after receiving the user's operation to delete the floating UI element, the mobile phone can delete the corresponding floating UI element and no longer display it on the mobile phone screen. The operation for deleting the floating UI element can be a long press operation, a double-click operation, or a drag operation on the floating UI element. The drag operation can be an operation of pressing and holding the floating UI element and quickly sliding it in one direction (such as up or down) (such as the sliding distance within a preset time meets a predetermined condition).
[0217] For another example, the duration of the floating UI element allowed to be displayed on the mobile phone screen may be unlimited (that is, it will always be displayed on the mobile phone screen if the user does not actively delete it), or it may be restricted. For example, the maximum existence time of the floating UI element may be set (user setting or system default setting). For example, the maximum existence time of the floating UI element can be set to X hours, then after the floating UI element is generated and displayed on the mobile phone screen, the mobile phone can start timing to determine the existence time of the floating UI element on the mobile phone screen. After the timing duration is equal to the maximum existence time of X hours, the mobile phone can automatically delete the floating UI element and no longer display it on the mobile phone screen. Alternatively, after the floating UI element is generated and displayed on the mobile phone screen, the mobile phone can start timing, and if no user operation on the floating UI element is received within X hours, the mobile phone will automatically delete the floating UI element and no longer display it on the mobile phone screen after the timing duration is equal to the maximum existence time of X hours. If the user operation on the floating UI element is received within X hours, the timing is restarted. In this embodiment, in order to facilitate the user to know the existence time of the floating UI element on the screen, after the floating UI element is generated and displayed on the mobile phone screen, the mobile phone can display corresponding prompt information during the time counting process to prompt the user the existence time of the floating UI element. Fig.28 As shown in (a) of FIG. 1 , the mobile phone displays a floating UI element 2801. The mobile phone may also display a ring bar 2802, which increases as the timing duration of the mobile phone for the floating UI element 2801 increases. When the timing duration is equal to the maximum existence time, the ring bar 2802 becomes a closed ring. When the timing duration is equal to the maximum existence time, the mobile phone deletes the floating UI element 2801 and no longer displays it on the mobile phone screen. For example, the mobile phone may display an animation of the floating UI element 2801 exploding (e.g., an animation similar to a bubble exploding), and then delete the floating UI element 2801 from the mobile phone screen.
[0218] For another example, an area for deleting floating UI elements may be pre-set. After the mobile phone receives the user's request to drag a floating UI element to the area and release the drag (this operation may be the fourth operation in the embodiment of the present application), the mobile phone may delete the floating UI element and no longer display it on the mobile phone screen. Fig.28As shown in (b) in the figure, the mobile phone displays a floating UI element 2803. After the mobile phone receives the user's operation of dragging the floating UI element 2803, it can display an animation of the floating UI element 2803 moving with the movement of the user's finger on the mobile phone screen. In addition, the mobile phone can also display a delete button 2804, and the area where the delete button 2804 is located is the area for deleting floating UI elements. If the mobile phone determines that the user drags the floating UI element 2803 to the area where the delete button 2804 is located and releases the drag, the mobile phone can delete the floating UI element 2803 and no longer display it on the mobile phone screen.
[0219] It should be noted that the above embodiments are illustrated by taking the floating UI element as a circular shape as an example. In some other embodiments, the floating UI element may also be a rectangle, a rounded rectangle, an ellipse, a polygon, or other irregular UI shapes. This embodiment does not impose any specific restrictions on the style of the floating UI element. The content in the floating UI element (or the elements that can be presented as floating UI elements) is also not limited to the text, picture, icon, card, list contact, video window, file resource, and other elements in this embodiment. Of course, the content in the floating UI element may also be a combination of these types of elements. In addition, the above embodiments are illustrated by taking the floating UI element displayed on the left or right edge of the screen as an example. In some other embodiments, the floating UI element may also be displayed at other positions of the screen, such as the top edge. This embodiment does not impose any specific restrictions on the display position (or layout position) of the floating UI element. The display position of the floating UI element may be the system default configuration, or a setting interface may be provided for the user to set. It can be seen that in the above embodiments, the floating UI element is illustrated by taking the example of being completely displayed on the edge of the screen in an opaque manner. In some other embodiments, combined with Fig.29 , floating UI elements, such as, Fig.29 The floating UI element 2901 shown in (a) can also be displayed in a semi-transparent manner, or as shown in FIG. Fig.29 As shown in (b) of FIG. 1 , the floating UI element 2902 may also be partially hidden at the edge of the screen, or as shown in FIG. Fig.29 As shown in (c) in FIG. 2 , the floating UI element 2903 is partially hidden at the edge of the screen. The way in which the floating UI element is displayed can be a system default configuration, or a setting interface can be provided for the user to set.
[0220] In addition, as mentioned above Figure 24-Figure 27In the example shown, a floating UI element group may be displayed with prompt information, such as a badge, for indicating the number of floating UI elements included in the floating UI element group. Whether to display a badge on a floating UI element group may be a system default configuration, or a setting interface may be provided for the user to set. For example, a setting interface may be provided for the user to set whether to display a badge on the floating UI element group for indicating the number of floating UI elements included in the floating UI element group, that is, to set the badge to be displayed or not. In some other embodiments, other badges may also be displayed for floating UI elements.
[0221] Such as Fig.30 As shown in (a) of the figure, for a floating UI element of a list contact person, such as Fig.30 In the floating UI element 3001 shown in (a), the mobile phone can display message reminder information on the floating UI element when receiving a message from the contact, such as Fig.30 As shown in (a) in FIG. 1 , the message reminder information may be displayed in the form of a corner mark 3002, and the number in the corner mark 3002 may be used to indicate the number of messages received. Of course, whether to display the message reminder information may be a system default configuration, or a setting interface may be provided for the user to set. If the message reminder information is displayed on the floating UI element when a message from a contact is received, the message reminder information may disappear and no longer be displayed after the user views it.
[0222] For another example, an application identifier, such as an application icon, may be displayed on the floating UI element to indicate the source of the floating UI element, so that the user can easily identify the original source of the floating UI element, that is, which application it comes from. Fig.30 The floating UI element 3003 shown in (b) of FIG. 3003 has an icon 3004 of an application indicating the source of the floating UI element 3003 displayed on it. Whether to display the identifier of the application indicating the source of the floating UI element can be a system default configuration, or a setting interface can be provided for the user to set. For example, the system default configuration displays the identifier of the application by default for the floating UI element of the list contact person. For another example, a setting interface can be provided for the user to set the identifier of the application to be displayed for one or more floating UI elements, and other types of floating UI elements are not displayed.
[0223] For another example, a symbol indicating the type of the floating UI element (such as a link) may be displayed on the floating UI element. For example, taking a floating UI element of a file resource type as an example, the mobile phone displays the following: Fig.30 The floating UI element 3005 shown in (c) has a symbol 3006 displayed on it for indicating the type of the floating UI element 3005 (eg, a link).
[0224] In some other embodiments, the corner marks in the above examples may appear at the same time, such as a floating UI element displaying a message reminder and an application logo indicating the source of the floating UI element. Fig.30 ) and a prompt message for indicating the number of floating UI elements included in the floating UI element group (as shown in (a) in FIG. Fig. 27 As shown in (a) in the figure). When the floating UI element group contains both message reminder information and prompt information, in order to distinguish them, the two types of information can be displayed in different positions of the floating UI element group, such as displaying the message reminder information in the upper left corner and the prompt information in the lower left corner. Furthermore, different display forms can be used to distinguish the two types of information. For example, if both types of information are presented in the form of superscripts, the colors of the superscripts corresponding to the message reminder information and the superscripts corresponding to the prompt information are different, or the shapes are different (such as one superscript is round and the other is square). In scenarios where the two types of information do not appear at the same time, the two types of information can also be distinguished. For details, please refer to the description of the scenario where the two types of information appear at the same time.
[0225] In addition, as described in the above embodiments, the floating UI element displayed on the screen can be dragged by the user to move on the screen. In some other embodiments, when the floating UI element moves on the mobile phone screen, the mobile phone can display a corresponding prompt (such as a prompt in the form of a corner mark) according to the current area where the floating UI element is located, to prompt the user whether the current area supports interaction. Fig.31 , take the floating UI element of the text class as an example. Fig.31 As shown in (a) of FIG. 1 , the mobile phone displays a floating UI element 3101. The user can press and move the floating UI element 3101 to trigger the floating UI element 3101 to move along with the movement of the user's finger. Fig.31 As shown in (b) of FIG. 3 , when the floating UI element 3101 moves to a non-interactive area, a corner mark 3102 is displayed on the floating UI element 3101, and the corner mark 3102 is used to prompt the user that the current area does not support interaction. Fig.31 As shown in (c) of FIG. 3 , when the floating UI element 3101 moves to an interactive area, such as the area where the input box 3103 is located, a corner mark 3104 is displayed on the floating UI element 3101, and the corner mark 3104 is used to prompt the user that the current area supports interaction. After the user releases the drag, the text carried by the floating UI element 3101 can be pasted into the input box 3103. Fig.31As shown in (d), when the floating UI element 3101 moves to the deletion area (the area may be predefined) 3106, a badge 3105 is displayed on the floating UI element 3101, and the badge 3105 is used to prompt the user that the floating UI element 3101 will be deleted after releasing the drag in the area.
[0226] It can be seen from the description of the above embodiments that in the scenario where an element is triggered to be presented as a floating UI element by dragging, after the user drags the element and releases the drag, the mobile phone can be triggered to present the element as a floating UI element on the mobile phone screen. In some other embodiments, a trigger area can also be pre-defined. When the user drags the element to the trigger area and releases the drag in the trigger area, the mobile phone can present the element as a floating UI element on the mobile phone screen. If the user drags the element to other areas and releases the drag, the mobile phone will not be triggered to perform the operation of presenting the element as a floating UI element.
[0227] By adopting the above technical solution, the mobile phone can display any element on the interface in the form of a floating UI element on the mobile phone screen according to the user operation. After that, no matter whether the user switches pages in the application or exits the application or opens other applications, the floating UI element can continue to be displayed on the mobile phone screen to realize cross-application information viewing, file transfer, information sharing and other operations. For example, floating UI elements of text, picture, card, file resource and video window classes support interactions such as sending to friends through instant messaging applications. Files, pictures, texts and URLs can be directly shared with the corresponding contacts by dragging them to the floating UI elements of the list contacts. URL floating UI elements support direct click jump. Floating UI elements representing specific contacts extracted from the contact lists of different applications support functions such as sending files and making calls according to specific business scenarios. In addition, floating UI elements are generated in combination with the existing interaction logic of the system.
[0228] Fig.32 The present invention provides a schematic diagram of a display device according to an embodiment of the present invention. The device can be applied to a terminal. Fig.32 As shown, the device may include: a display unit 3201 and an input unit 3202.
[0229] The display unit 3201 is used to display a first interface of the application on the display screen of the terminal, where the first interface includes a first element.
[0230] The input unit 3202 is used to receive a drag operation on the first element.
[0231] The display unit 3201 is further configured to display an animation of the first element moving along with the drag operation on the display screen of the terminal.
[0232] The input unit 3202 is also used to receive a drag-and-release operation.
[0233] The display unit 3201 is further used to display a first floating element on the display screen of the terminal, and the content of the first floating element is the same as the content of the first element.
[0234] Furthermore, the first interface may also include a second element within a preset range of the first element; and the first floating element also includes the content of the second element.
[0235] Furthermore, the first element may be one or more of a text element, a URL, a picture element, an icon element, a card element, a list contact element, a video window element or a file resource element.
[0236] Furthermore, the input unit 3202 is also used to receive a first operation.
[0237] The display unit 3201 is further used to display the elements supporting floating display in the first interface in a first form, and display the elements not supporting floating display in a second form; the first element is an element supporting floating display, and the first form is different from the second form.
[0238] Further, the first operation may be an operation on a predetermined function control; or the first operation may be an operation on an element in the first interface that supports floating display.
[0239] Furthermore, the display unit 3201 is also used to display a second interface on the display screen of the terminal, and the second interface includes a third element.
[0240] The input unit 3202 is further used to receive a second operation on the third element.
[0241] The display unit 3201 is further used to display a second floating element on the display screen of the terminal, the content of the second floating element is the same as the content of the third element, and the second floating element and the first floating element are displayed on the display screen of the terminal according to a preset arrangement rule. The second operation may include a drag operation and a drag-and-release operation, or when the first element is a text element or a URL, the second operation may also be a copy operation.
[0242] Furthermore, the display device may also include: a determining unit 3203, configured to determine that the number of floating elements displayed on the display screen of the terminal is less than N, where N is the maximum number of floating elements allowed to be displayed simultaneously on the display screen.
[0243] Furthermore, the display device may also include: a deleting unit 3204, which is used to delete the first floating UI element arranged according to a preset arrangement rule among the N floating elements when the determination unit determines that the number of floating elements displayed on the terminal display screen is equal to N, and display the second floating element.
[0244] Furthermore, the determining unit 3203 is further configured to determine that there is no floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal.
[0245] Furthermore, the display unit 3201 is also used to merge the second floating element and the third floating element into a floating element group and display it on the display screen of the terminal when the determination unit 3203 determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal.
[0246] Furthermore, the input unit 3202 is further used to receive a third operation on the floating element group; the display unit 3201 is used to display the second floating element and the third floating element on the display screen of the terminal.
[0247] Furthermore, the input unit 3202 is further configured to receive a fourth operation on the first floating element, and the deleting unit 3204 is further configured to delete the first floating element. Alternatively, the deleting unit 3204 is further configured to delete the first floating element after the first floating element is displayed on the display screen of the terminal for a preset time.
[0248] Furthermore, one or more of the following logos are displayed on the first floating element: an application logo, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element.
[0249] Furthermore, when the first element is a text element, a URL, a picture element or a file resource element, the display unit 3201 is also used to display an interface including an input box.
[0250] The input unit 3202 is further used to receive a drag operation on the first floating element and release the drag operation at the input box position.
[0251] The display device may further include: a filling unit 3205, configured to fill the content corresponding to the first floating element into the input box.
[0252] Furthermore, in the case where the first element is an icon-type element or a list-type element, the input unit 3202 is further used to receive a fifth operation on the first floating element.
[0253] The display device may further include: an execution unit 3206, configured to execute a first function, where the first function is the same as the function executed after the terminal receives an operation on the first element by the user.
[0254] In another possible implementation, when the first element is a list contact element, the input unit 3202 is further used to receive a drag operation on text, URL or image in the interface, and release the drag operation at the first floating element position.
[0255] The display device may further include: a sending unit 3207, configured to send text, URL address or picture to a terminal of a contact corresponding to the first floating element.
[0256] In another possible implementation, the display unit 3201 is used to display a first interface on a display screen of the terminal, where the first interface includes a first element, where the first element is a text element or a URL. The input unit 3202 is used to receive a copy operation on the first element. The display unit 3201 is also used to display a first floating element on the display screen of the terminal, where the content of the first floating element is the same as the content of the first element.
[0257] The present application also provides a display device, which can be applied to an electronic device, such as the terminal in the above embodiment. The device may include: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions so that the display device implements the various functions or steps executed by the terminal (such as a mobile phone) in the above method embodiment.
[0258] The embodiment of the present application provides an electronic device (such as the above-mentioned terminal), which includes a display screen, one or more processors and a memory; the display screen, the processor and the memory are coupled; the memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the electronic device, the electronic device implements the various functions or steps performed by the terminal (such as a mobile phone) in the above-mentioned method embodiment. Of course, the electronic device includes but is not limited to the above-mentioned display screen, memory and one or more processors. For example, the structure of the electronic device can refer to Figure 3 The structure of the mobile phone shown.
[0259] The present application also provides a chip system, which can be applied to the terminal in the above embodiment. Fig.33As shown, the chip system includes at least one processor 3301 and at least one interface circuit 3302. The processor 3301 may be a processor in the above-mentioned terminal. The processor 3301 and the interface circuit 3302 may be interconnected via a line. The processor 3301 may receive and execute computer instructions from the memory of the above-mentioned terminal via the interface circuit 3302. When the computer instructions are executed by the processor 3301, the terminal may execute the various steps executed by the mobile phone in the above-mentioned embodiment. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.
[0260] The embodiment of the present application also provides a computer-readable storage medium for storing computer instructions executed by the above-mentioned terminal.
[0261] The embodiment of the present application also provides a computer program product, including computer instructions executed by the above-mentioned terminal.
[0262] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0263] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0264] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0265] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0266] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0267] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims
1. A method for displaying interface elements, applied to a terminal, characterized in that: The method comprises: The terminal displays a first interface of the application on a display screen of the terminal, where the first interface includes a first element; After receiving the drag operation on the first element, the terminal displays an animation of the first element moving along with the drag operation on a display screen of the terminal; After receiving the drag-and-release operation, the terminal displays a first floating element on a display screen of the terminal, wherein the content of the first floating element is the same as the content of the first element; The terminal displays a second interface on a display screen of the terminal, where the second interface includes a third element; The terminal determines that the number of floating elements displayed on the display screen of the terminal is equal to N, where N is the maximum number of floating elements allowed to be displayed simultaneously on the display screen; After receiving a drag operation and a drag-release operation on the third element, the terminal determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal. The terminal merges the second floating element and the third floating element into a floating element group and displays it on the display screen of the terminal; the content of the second floating element is the same as the content of the third element.
2. The method according to claim 1, characterized in that The first interface also includes a second element within a preset range of the first element; The first floating element also includes the content of the second element.
3. The method according to claim 1 or 2, characterized in that: The first element is one or more of a text element, a uniform resource locator (URL), a picture element, an icon element, a card element, a list contact element, a video window element or a file resource element.
4. The method according to claim 1 or 2, characterized in that: After the terminal displays the first interface on the display screen, the method further includes: After receiving the first operation, the terminal displays the elements in the first interface that support floating display in a first form, and displays the elements that do not support floating display in a second form; the first element is an element that supports floating display, and the first form is different from the second form.
5. The method according to claim 4, characterized in that The first operation is an operation on a predetermined function control; or the first operation is an operation on an element in the first interface that supports floating display.
6. The method according to claim 1 or 2, characterized in that: The method further comprises: When the terminal determines that there is no floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal, the terminal displays the second floating element on the display screen of the terminal, and the second floating element and the first floating element are displayed on the display screen of the terminal according to a preset arrangement rule.
7. The method according to claim 6, characterized in that The method further comprises: The first floating element arranged according to the preset arrangement rule among the N floating elements is deleted, and the second floating element is displayed.
8. The method according to claim 1 or 2, characterized in that: The method further comprises: After receiving the third operation on the floating element group, the terminal displays the second floating element and the third floating element on a display screen of the terminal.
9. The method according to claim 1 or 2, characterized in that: The method further comprises: After receiving the fourth operation on the first floating element, the terminal deletes the first floating element; or, The terminal deletes the first floating element after the first floating element is displayed on a display screen of the terminal for a preset time period.
10. The method according to claim 1 or 2, characterized in that: One or more of the following identifiers are displayed on the first floating element: an identifier of the application, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element.
11. The method according to claim 1 or 2, characterized in that: In the case where the first element is a text element, a URL, a picture element or a file resource element, after the terminal displays the first floating element on the display screen, the method further includes: The terminal displays an interface including an input box; After receiving the drag operation on the first floating element and releasing the drag operation at the input box position, the terminal fills the content corresponding to the first floating element into the input box.
12. The method according to claim 1 or 2, characterized in that: In the case where the first element is an icon-type element or a list-type element, after the terminal displays the first floating element on the display screen, the method further includes: After receiving the fifth operation on the first floating element, the terminal executes a first function, where the first function is the same as the function executed after the terminal receives the user's operation on the first element.
13. The method according to claim 1 or 2, characterized in that: In the case where the first element is a list-contact-human element, after the terminal displays the first floating element on a display screen, the method further includes: After receiving a drag operation on text, URL or picture in the interface and releasing the drag operation at the position of the first floating element, the terminal sends the text, URL or picture to the terminal of the contact corresponding to the first floating element.
14. A method for displaying interface elements, applied to a terminal, characterized in that: The method comprises: The terminal displays a first interface on a display screen of the terminal, wherein the first interface includes a first element, and the first element is a text element or a uniform resource locator URL; After receiving the copy operation on the first element, the terminal displays a first floating element on a display screen of the terminal, wherein the content of the first floating element is the same as the content of the first element; The terminal displays a second interface on a display screen of the terminal, where the second interface includes a third element; The terminal determines that the number of floating elements displayed on the display screen of the terminal is equal to N, where N is the maximum number of floating elements allowed to be displayed simultaneously on the display screen; After receiving a copy operation on the third element, the terminal determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal. The terminal merges the second floating element and the third floating element into a floating element group and displays it on the display screen of the terminal; the content of the second floating element is the same as the content of the third element.
15. The method according to claim 14, characterized in that After the terminal displays the first floating element on a display screen, the method further includes: The terminal displays an interface including an input box; After receiving the drag operation on the first floating element and releasing the drag operation at the input box position, the terminal fills the content corresponding to the first floating element into the input box.
16. A display device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to enable the display device to implement the method as described in any one of claims 1-13, or to implement the method as described in any one of claims 14-15 when executing the instructions.
17. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method according to any one of claims 1 to 13, or implements the method according to any one of claims 14 to 15.
18. A computer program product comprising instructions, characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 13, or execute the method according to any one of claims 14 to 15.
