Interface display method and apparatus

Abstract

Embodiments of the present application relate to the technical field of terminals, and provide an interface display method and apparatus. The method comprises: upon receiving a first operation for a first interface, displaying a second interface; and upon receiving a first message acquired by means of a microphone, displaying a third interface, wherein the third interface comprises: a first window, and the first window comprises: a second message, a third message, and a fourth message. In this way, on the basis of an operation of a user inputting the first message by means of voice, an electronic device can conveniently push a message related to the first message and the third message (such as a related picture in an interface), so that while the user performs voice interaction with a voice assistant, the voice assistant can provide a more accurate reply message on the basis of the voice input and in combination with the content in the current interface, thereby improving use experience of the user.

Description

Interface display method and device TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to an interface display method and device. BACKGROUND

[0002] With the popularity and development of the Internet, the functional requirements of people for electronic devices are increasingly diversified. For example, in order to improve the use requirements of users for multiple services, an electronic device can support the flow of target content between multiple services. The target content can include text, pictures, videos, or files, etc.

[0003] Generally, when a user needs to query a restaurant mentioned in a social application, the user can perform a series of operations such as copying the name of the restaurant in the social application, opening a review application from the background or desktop, clicking a search bar in the review application, pasting the name of the restaurant, clicking search, etc. to view the details of the restaurant.

[0004] However, the above operations are relatively cumbersome, which affects the user experience.

[0005] SUMMARY

[0006] Embodiments of the present application provide an interface display method and device, applied to the technical field of terminals, for simplifying user operations and improving user experience.

[0007] In a first aspect, an interface display method is provided, which includes: displaying a first interface, the first interface including a first element, the first element including one or more of the following: text, pictures, videos, or files; after receiving a first operation on the first interface, displaying a second interface, the second interface including: display content of the first interface, an identifier of a first voice assistant, and a first prompt information, the first prompt information being used to prompt that a microphone is receiving sound; after receiving a first message obtained through the microphone, displaying a third interface, wherein the third interface includes: a first window, the first window including: a second message, a third message, and a fourth message, the first message being a message related to the first element, the second message including text corresponding to the first message, the third message including a picture corresponding to the first element, and the fourth message being a reply message provided based on the second message and the third message.

[0008] In this way, the electronic device can conveniently push a message related to the first message and the third message (such as a related picture in the interface) based on the operation of the user inputting the first message by voice, so that the user can interact with the voice assistant by voice while the voice assistant can provide a more accurate reply message based on the voice input and in combination with the content in the current interface, thereby improving the user experience.

[0009] In a possible implementation, after displaying the second interface, the method further includes: receiving a second operation, the second operation being an operation on the first element in the second interface, and the third message being a picture generated based on the second operation.

[0010] The second operation is a target operation (such as a circle selection operation) on FIG. 3A, and the third message can be a picture generated based on the circle selection operation.

[0011] In this way, the user can complete input of the user intention in the voice assistant through the operation of speaking while drawing, so as to improve flexibility of the user in using the voice assistant.

[0012] In a possible implementation, before displaying the third interface, the method further includes: during receiving the second operation, displaying a motion track of the second operation in the second interface; when the first message ends at a first time and the second operation does not end, continuing to display the motion track of the second operation in the second interface, and the third interface being an interface displayed after the second operation ends at a second time, the first time being earlier than the second time.

[0013] The motion track of the second operation can be the circle selection track 30 in FIG. 3B.

[0014] It can be understood that the embodiments of the present application do not limit the order of the voice input and the circle selection operation of the user. For example, in the scenario of speaking while drawing, the user can end the voice message input first and then end the circle selection operation, and the third interface is displayed after the circle selection operation ends, so as to improve flexibility of the voice assistant.

[0015] In a possible implementation, before displaying the third interface, the method further includes: during receiving the second operation, displaying a motion track of the second operation in the second interface; when the second operation ends at a third time and the first message does not end, continuing to acquire the first message through the microphone, and the third interface being an interface displayed after the first message ends at a fourth time, the third time being earlier than the fourth time.

[0016] It can be understood that the embodiments of the present application do not limit the order of the voice input and the circle selection operation of the user. For example, in the scenario of speaking while drawing, the user can end the circle selection operation first and then end the voice input operation, and the third interface is displayed after the voice input operation ends, so as to improve flexibility of the voice assistant.

[0017] In a possible implementation, the second interface further includes a first button, when the second operation ends at the third moment and the first message does not end, the fourth interface is further displayed after receiving an operation on the first button, and the fourth interface includes a second window, and the second window includes the third message and the first prompt information, or the second window includes the third message and a fifth message, the fifth message is a text message generated based on the first message that does not end.

[0018] It can be understood that, in a scenario where the electronic device does not interrupt the voice message based on the operation on the first button, the second window can include the third message and the first prompt information. Or, in a scenario where the electronic device interrupts the voice message based on the operation on the first button, the second window can include the third message and the fifth message. The fifth message can be, for example, "This place..." and the like.

[0019] The first prompt information can be prompt information displayed in the input box 321 of FIG. 3G, which is used to prompt that the voice assistant is in a listening state.

[0020] In a possible implementation, before the third interface is displayed, the method further includes: displaying a motion trail of the second operation in the second interface during receiving the second operation, and the third interface is an interface displayed when the second operation and the first message end at the same time.

[0021] It can be understood that the embodiments of the present application do not limit the order of the user's speech input and the circle selection operation. For example, in the scenario of saying while drawing, the user can end the circle selection operation and the speech input operation at the same time, and the third interface is displayed, so as to improve the flexibility of the voice assistant.

[0022] In a possible implementation, the second interface further includes a first button, and before the third interface is displayed, the method further includes: displaying a motion trail of the second operation in the second interface, and the third interface is an interface displayed after receiving an operation on the first button.

[0023] The first button can be the button 306 in FIG. 3A. In this way, the user can also conveniently pull up the third interface through the button 306.

[0024] In a possible implementation, when the second message and the third message belong to messages in the first scenario, the fourth message is a reply message provided by a second voice assistant based on the second message and the fourth message, the second voice assistant is used to process the dialogue message in the first scenario, and the fourth message is surrounded by an identifier of the second voice assistant.

[0025] The first scenario can be a life information scenario, and the second voice assistant can be a life service assistant. The identifier of the second voice assistant can be an identifier of the life service assistant, such as a service identifier 421 and / or a service name 422.

[0026] In this way, the user can quickly learn which voice assistant provides the message through the identifier of the second voice assistant.

[0027] In a possible implementation, after the third interface is displayed, the method further includes: after receiving a third operation on the first element in the second interface, displaying, in the first window, a sixth message and a seventh message, wherein the seventh message includes an identifier of a third voice assistant, and the sixth message is a picture generated based on the third operation, and the seventh message is a reply message provided by the third voice assistant based on the sixth message when the sixth message belongs to a message in a second scenario, and the third voice assistant is used to process a conversation message in the second scenario.

[0028] The third operation can be a circle selection operation of the user on a region in which the teacup in FIG. 5A is located.

[0029] The sixth message can be the message 502 in FIG. 5C or FIG. 5D, and the seventh message can include the message 504 and / or the message 505 in FIG. 5D.

[0030] The second scenario can be a life information scenario, and the third voice assistant can be a life service assistant.

[0031] In this way, the electronic device can provide a more appropriate reply message in the first window based on the content re-selected by the user, to improve the flexibility of the user in using the voice assistant.

[0032] In a possible implementation, the first window further includes a second button. After the third interface is displayed, the method further includes: after receiving an operation on the second button, displaying a fifth interface, the fifth interface including a third button related to the second voice assistant; after receiving an operation on the third button, displaying a sixth interface, the sixth interface including a third window used to process a conversation message in the first scenario; and after receiving an operation of inputting a second message and a third message into the third window, displaying, in the third window, the second message, the third message, and a fourth message.

[0033] The second button can be the button 308 in FIG. 3C. The fifth interface can be the interface shown in FIG. 4C, the third button can be the button 410, the sixth interface can be the interface shown in FIG. 4I, and the third window can be the conversation window 428.

[0034] In this way, the electronic device can make targeted demand questions in the third window after detecting that the user enters the third window corresponding to the second voice assistant, and the second voice assistant can also provide more accurate message replies.

[0035] In a possible implementation, the first window further includes a fourth button for inputting a second element in the first window, the second element including: a picture, a video, or a document, and after displaying the third interface, further including: displaying one or more of the following in the first window after receiving an operation on the fourth button: a button for viewing the picture or the video, a button for starting the camera, and a button for viewing the document.

[0036] The fourth button can be the button 311 in FIG. 3C. As shown in FIG. 6C, the button for viewing the picture or the video can be the button 604 in FIG. 6C, the button for starting the camera can be the button 603, and the button for viewing the document can be the button 605.

[0037] In this way, the user can also input more elements into the first window through one or more of the button for viewing the picture or the video, the button for starting the camera, and the button for viewing the document.

[0038] In a possible implementation, after displaying the first interface, further including: displaying a seventh interface after receiving a fourth operation of dragging the first element to the edge of the screen, the seventh interface including: the display content in the first interface, and a fifth button for starting the first voice assistant; in a case where the fourth operation continues to act on the first element, displaying the first element in the first window after receiving an operation of dragging the fourth element to a position where the fifth button is located.

[0039] The fourth operation of dragging the first element to the edge of the screen can be an operation of dragging the text information in the dashed box 102 in FIG. 1B to the edge of the screen by the user.

[0040] The seventh interface can be the interface shown in FIG. 6A, and the fifth button can be the button 601. When the first element is text, the first element displayed in the first window can be the message 620 in FIG. 6B.

[0041] In this way, the user can quickly pull up the first window through the fourth operation, and ask questions about the content in the interface in the first window.

[0042] In a second aspect, an embodiment of the present application provides an interface display apparatus. The interface display apparatus can be an electronic device, or a chip or chip system in the electronic device. The interface display apparatus can include a display unit and a processing unit. When the interface display apparatus is an electronic device, the display unit can be a display screen. The display unit is configured to perform the displaying step, so that the electronic device implements an interface display method described in the first aspect or any possible implementation of the first aspect. When the interface display apparatus is an electronic device, the processing unit can be a processor. The interface display apparatus can further include a storage unit, which can be a memory. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the electronic device implements an interface display method described in the first aspect or any possible implementation of the first aspect. When the interface display apparatus is a chip or chip system in the electronic device, the processing unit can be a processor. The processing unit executes the instructions stored in the storage unit, so that the electronic device implements an interface display method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit (for example, a register, a cache, etc.) in the chip, or a storage unit (for example, a read-only memory, a random access memory, etc.) outside the chip in the electronic device.

[0043] Specifically, the display unit is configured to display a first interface, the first interface including a first element, the first element including one or more of the following: a text, a picture, a video, or a file; after the processing unit receives a first operation on the first interface, the display unit is further configured to display a second interface, the second interface including: display content of the first interface, an identifier of the first voice assistant, and first prompt information, the first prompt information being used to prompt that the microphone is receiving sound; after the processing unit receives a first message acquired through the microphone, the display unit is further configured to display a third interface, the third interface including: a first window, the first window including: the second message, the third message, and the fourth message, the first message being a message related to the first element, the second message including text corresponding to the first message, the third message including a picture corresponding to the first element, and the fourth message being a reply message provided based on the second message and the third message.

[0044] In a third aspect, an embodiment of the present application provides an electronic device. The electronic device includes one or more processors and a memory. The memory is coupled to the one or more processors, and is configured to store computer program codes including computer instructions. The one or more processors invoke the computer instructions to cause the electronic device to perform the method described in the first aspect or any possible implementation of the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which includes computer instructions. When the computer instructions are run on an electronic device, the electronic device executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program product includes computer program code, when the computer program code is run on an electronic device, the electronic device executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0047] In a sixth aspect, the present application provides a chip system applied to an electronic device. The chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0048] In a possible implementation, the chip system described in the present application further includes at least one memory in which instructions are stored. The memory can be a storage unit inside the chip system, for example, a register, a cache, etc., or a storage unit of the chip system (for example, a read-only memory, a random access memory, etc.).

[0049] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding possible implementation manner are similar, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0050] FIGS. 1A-1D are schematic diagrams of a scenario provided by an embodiment of the present application;

[0051] FIG. 2 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application;

[0052] FIGS. 3A-3G are schematic diagrams of an interface provided by an embodiment of the present application;

[0053] FIGS. 4A-4K are schematic diagrams of an interface provided by an embodiment of the present application;

[0054] FIGS. 5A-5D are schematic diagrams of an interface provided by an embodiment of the present application;

[0055] FIGS. 6A-6F are schematic diagrams of an interface provided by an embodiment of the present application;

[0056] FIG. 7 is a flowchart of an interface display method provided by an embodiment of the present application;

[0057] FIG. 8 is a schematic diagram of a hardware structure of another electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] For the purpose of clearly describing the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0059] 1. Voice assistant

[0060] The voice assistant can be in the form of a service or an application in the electronic device.

[0061] The voice assistant can also be referred to as a first voice assistant, a YOYO assistant, an intelligent dialogue, an artificial intelligence (AI) assistant, or other names, which are not limited in the embodiments of the present application. In the interface related to the voice assistant, the voice assistant will be presented as YOYO assistant in the interface, which will not be described in detail hereinafter.

[0062] For example, after the user wakes up the voice assistant, the user can input voice information or text information of "check the weather of A city today" to the electronic device, and the voice assistant can broadcast the weather of A city, so that the user can know the weather of A city without opening the weather application.

[0063] The voice assistant has three states: a silent state, a listening state (or a hearing state), and a thinking state. In the silent state, the voice assistant does not receive sound, and the microphone can be in an off state; in the listening state, the voice assistant starts to receive sound, and the microphone can be in an on state and receive voice information of the user; in the thinking state, the voice assistant ends to receive sound and replies to the message input by the user.

[0064] 2. Intelligent agent

[0065] The intelligent agent can also be referred to as a second voice assistant.

[0066] The intelligent agent and the voice assistant can be parallel services. Alternatively, the intelligent agent can also provide services in different scenarios in the voice assistant, and the intelligent agent can also be referred to as a sub-service, a sub-scenario, an artificial intelligence (AI) skill, an AI model, an AI service, or other names, which are not limited in the embodiments of the present application.

[0067] The intelligent agent can be divided according to different use scenarios. For example, the intelligent agent can include one or more of the following: a smart travel assistant applied to a travel scenario, a life service assistant applied to a life information scenario, or a smart video editing assistant applied to a video editing scenario, and the like.

[0068] The smart travel assistant can implement functions such as flight query, ticket booking, and itinerary planning. The life service assistant can implement functions related to the user's daily life, such as restaurant query, schedule creation, and schedule reminder. The smart video editing assistant can implement functions such as automatically generating short videos based on pictures, videos, and audio content in the electronic device within a certain period of time.

[0069] The intelligent agent can be a service developed by the manufacturer of the electronic device, or a service generated by the manufacturer of the electronic device in cooperation with a third-party application. The use method of the intelligent agent can be referred to the description in FIGS. 4C-4D.

[0070] It can be understood that the intelligent agent name provided in the embodiments of the present application and the functions allowed to be implemented by the intelligent agent are only used as an example and do not constitute a limitation on the embodiments of the present application.

[0071] 3, dialogue window

[0072] The dialogue window can be a window for interacting with the user. The dialogue window can also be referred to as a window. The dialogue window can include dialogue messages. For example, the dialogue messages can include a demand message input by the user (e.g., message 1) and a reply message provided by the electronic device based on the demand message (e.g., message 2).

[0073] For example, the message 1 can include a text message input by the user through a text box, a voice message input by the user, or a message selected by the user in any application interface. The message type of the message 1 or the message 2 can be text, picture, voice, video, or file, and the message type of the message 1 and the message 2 is not limited in the embodiments of the present application.

[0074] The dialogue window can include a first dialogue window and a second dialogue window.

[0075] The first dialogue window can be a dialogue window pulled up after the wake-up voice assistant, or understood as a dialogue window corresponding to the voice assistant. The user can interact with the voice assistant through the first dialogue window. For example, the first dialogue window can be the dialogue window 305 in FIG. 3C.

[0076] The second dialogue window can be a dialogue window pulled up after entering any intelligent agent, or understood as a dialogue window corresponding to the intelligent agent. The user can interact with any intelligent agent through the second dialogue window.

[0077] It can be understood that in the case of including multiple agents in the voice assistant, each agent can correspond to a second dialog window. For example, the second dialog window can include: the dialog window 411 in FIG. 4D, or the dialog window 428 in FIG. 4I, etc.

[0078] 4. A user interface (UI) component

[0079] The UI component can be a basic element constituting a UI. In the case that the electronic device displays any UI component, the user can achieve interaction with the electronic device by triggering the UI component.

[0080] The UI component can include one or more of the following: a window, a button, a list, a card, a multitasking selector, etc.

[0081] The window can be understood as an independent view, and the window can generally include a title bar, a border, a content area, etc. The button can be understood as an element that allows the user to click, and the button can be used to trigger a certain operation or event. The list can be used to display a group of items or entries, and the list usually appears in a vertical or horizontal arrangement, and each item can include text, icons, or other UI elements. The card can be understood as a visual container for grouping and displaying related information. The multitasking selector can allow the user to switch between multiple tasks (or buttons), for example, the multitasking selector can include a text box or a button component.

[0082] It can be understood that the dialog window described in the embodiments of the present application can display one or more UI elements. For example, the process of displaying a window in the dialog window can refer to the description of message 416 in FIG. 4F.

[0083] 5. An electronic device

[0084] The electronic device of the embodiments of the present application can include a handheld device with display function, a vehicle-mounted device, etc. For example, some electronic devices are: a mobile phone, a tablet computer, a palm computer, a notebook computer, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc., and the embodiments of the present application are not limited thereto. Among them, the wearable device can include one or more of the following: a smart watch, smart glasses, a smart bracelet, smart jewelry, etc.

[0085] The electronic device in the embodiments of the present application can also be referred to as: a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent, or a user apparatus, etc.

[0086] 6. Other terms

[0087] In the embodiments of the present application, the same items or similar items with basically the same functions and roles are distinguished by using "first", "second", etc. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit the sequence. Those skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different.

[0088] It should be noted that in the embodiments of the present application, "exemplary" or "for example" is used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0089] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0090] The flow process of the target content between multiple services will be described below with reference to the corresponding embodiments of FIGS. 1A-1D. The target content can include text, pictures, videos, or files, etc.

[0091] FIGS. 1A-1D are schematic diagrams of a scenario provided by the embodiments of the present application. In the embodiments of the present application, an electronic device is taken as an example of a mobile phone, and the example does not constitute a limitation on the embodiments of the present application.

[0092] In response to the operation of the user opening the social application, the electronic device displays an interface as shown in FIG. 1A, which can be the home page of the social application. In FIG. 1A, the thumbnail of the note (such as thumbnail 101) and the function buttons located at the bottom of the interface can be displayed.

[0093] The thumbnail 101 can be the thumbnail of any picture in the note, such as the thumbnail of the first picture in the note.

[0094] The lower side of the thumbnail 101 can display one or more of the following: the title of the note (such as "A City! Landscape site near the exhibition center!"), the publisher of the note (such as "user 101"), or the number of likes of the note (such as 100 likes), etc.

[0095] The function buttons can include, from left to right, a button for viewing the home page, a button for viewing the shopping page, a button for creating a note, a button for viewing messages, and a button for viewing personal information (such as displayed as "I" word), etc. Among them, the button for viewing the home page is in the selected state, and the selected state can be the state of bold text and / or enlarged text, etc.

[0096] In response to the user performing the operation of clicking the thumbnail 101 in FIG. 1A, the electronic device displays an interface as shown in FIG. 1B, which can be a note detail interface. In FIG. 1B, picture information and / or text information, etc. can be displayed, such as the picture 103 and the text information located below the picture 103. The text information can include the title of the note and the body of the note, etc. The body of the note can include content such as "ABC Store (Exhibition Center Store) Afternoon Tea!".

[0097] In FIG. 1B, the identifier 110 can also be displayed, which can be used to indicate the positional relationship between the current picture (such as the picture 103) and other pictures in the note. The identifier 110 can be composed of multiple hollow or solid circles. For example, in the case where the identifier 110 includes three circles, the first circle is a solid circle, and the remaining two circles are hollow circles, the identifier 110 can indicate that the picture 103 is the first picture in the note. The user can view other pictures in the note by sliding the picture 103.

[0098] Optionally, in the case where the note contains N pictures, the picture 103 can also display "1 / N" or the like. 1 / N can be understood as that the note application contains N pictures, the picture 103 is the first picture in the N pictures, and N is an integer greater than 1.

[0099] In response to the user performing the operation of selecting the text information in the dashed box 102 in FIG. 1B and dragging the text information in the dashed box 102 to the right edge of the interface (at this time the user does not release the hand), the electronic device displays an interface as shown in FIG. 1C. In response to the user dragging the text information in the dashed box 102 to the position where the button 107 is located based on the dragging operation (at this time the user releases the hand), the electronic device displays an interface as shown in FIG. 1D. It can be understood that the dashed box 102 is only used to illustrate the text information selected by the user in FIG. 1B, and the dashed box 102 is not displayed in FIG. 1B.

[0100] In FIG. 1C, the picture 104, the thumbnail 106, and the service list 105 can be displayed.

[0101] The picture 104 can be a picture obtained by performing operations such as taking a screenshot, reducing, and rotating the content in FIG. 1B.

[0102] The thumbnail 106 can be a picture obtained by scaling the text information selected by the user, and the thumbnail 106 can display the text "ABC Store (Exhibition Center Store)".

[0103] The service list 105 can be services pushed by the electronic device based on the text information (e.g., the text in the dashed box 102) selected by the user. As can be seen from top to bottom in the service list 105, the service list 105 can include one or more of the following buttons: a button for a favorite space service, a button for a send to friend service, a button for a new schedule service, a button for a to-do service, a button for a store to cloud service, or a button for a review service 107, etc.

[0104] The favorite space service can be used to collect the text information in the dashed box 102, the send to friend service can be used to send the text information in the dashed box 102 to a friend in a chat application, the new schedule service can be used to create a to-do item related to the text information in the dashed box 102, the store to cloud service can be used to store the text information in the dashed box 102 to a cloud, and the review service can be used to search for a restaurant related to the text information in the dashed box 102, and to implement functions such as restaurant information query and restaurant reservation.

[0105] As can be seen in FIG. 1D, the note detail interface (i.e., the note detail interface shown in FIG. 1B) and the floating window 108 can be displayed. The floating window 108 can display a search interface and control buttons at the top of the floating window 108.

[0106] The search interface can be an interface obtained by the electronic device searching for “ABC Store (Exhibition Center Store)” in the review service, and the search interface can display a search box 109 and restaurant information. The search box 109 can display the text “ABC Store (Exhibition Center Store)” therein.

[0107] The control buttons at the top of the floating window 108 can include, from left to right, a full screen display button, a minimize button, a drag button, and a close button.

[0108] It can be understood that the electronic device can query a restaurant mentioned in a social application based on the descriptions of FIGS. 1A-1D above. Alternatively, the electronic device can also query a restaurant mentioned in a social application based on other manners. For example, in response to the user performing the operation of FIG. 1B, the electronic device can identify the content in FIG. 1B and mark the text information or picture information in FIG. 1B. In response to the user performing a series of operations of copying “ABC Store (Exhibition Center Store)”, opening an application interface corresponding to the review service, and pasting “ABC Store (Exhibition Center Store)” in the application interface corresponding to the review service, the electronic device can display restaurant information related to “ABC Store (Exhibition Center Store)”.

[0109] However, the above operations are relatively cumbersome and affect the user experience. Furthermore, when the user needs to further query a route from a location of the user to a location of the restaurant, the user needs to perform a series of operations of copying the address of the restaurant and searching for the address of the restaurant in a navigation application, which is relatively cumbersome.

[0110] Therefore, the interface display method provided in the embodiments of the present application enables the electronic device to push a reply message related to the mentioned text and picture in the voice input based on the voice input of the user for the current interface after starting the voice assistant, so that the voice assistant can provide a more accurate reply message based on the voice input and in combination with the content in the current interface while the user is performing voice interaction with the voice assistant, thereby improving the user experience.

[0111] In order to better understand the embodiments of the present application, the structure of the electronic device of the embodiments of the present application is introduced below. For example, FIG. 2 is a schematic diagram of the hardware structure of an electronic device provided in the embodiments of the present application.

[0112] The electronic device can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, an indicator 192, a camera 193, and a display screen 194, etc.

[0113] In the embodiments of the present application, the sensor module 180 can include a touch sensor 180A, which can be used to receive a trigger operation of the user for the electronic device.

[0114] The sensor module 180 can also include one or more of the following: a proximity light sensor and an ambient light sensor, a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, or a bone conduction sensor, etc. (not shown in FIG. 2), which are not specifically limited in the embodiments of the present application.

[0115] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0116] The processor 110 can include one or more processing units, which can implement the steps of detecting the air gesture of the user and controlling the device to execute corresponding instructions based on the air gesture of the user in the power supply method provided in the embodiments of the present application.

[0117] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like.

[0118] The charging management module 140 is configured to receive a charging input from a charger. The power management module 141 is configured to connect the charging management module 140 and the processor 110.

[0119] The wireless communication function of the electronic device can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0120] The electronic device can implement the display function by the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.

[0121] The display screen 194 is configured to display images and videos, and can display the interfaces shown in FIGS. 1A-1D and FIGS. 3A-6F described in the embodiments of the present application.

[0122] The electronic device can implement the photographing function by the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0123] The camera 193 is configured to capture still images or videos.

[0124] The external memory interface 120 can be configured to connect an external memory card. The internal memory 121 can be configured to store computer executable program codes, and the executable program codes include instructions.

[0125] The electronic device can implement the audio function by the audio module 170, the loudspeaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, the electronic device can play music, record sound, and the like.

[0126] The microphone 170C, also referred to as a “microphone”, “sound transducer”, is configured to convert a sound signal into an electrical signal. When a user inputs voice information to the electronic device through the microphone, the user can speak close to the microphone 170C by the mouth, and the electronic device can collect a sound signal through the microphone 170C, and can also convert the sound signal into text information and display the text information on the display screen.

[0127] The touch sensor 180A can be disposed on the display screen 194, and the touch sensor 180A and the display screen 194 can form a touch screen, also referred to as a “touch panel”. The touch sensor 180A can detect a target operation or other trigger operation of the user on the electronic device.

[0128] The keys 190 include a power key, a volume key, and the like.

[0129] The software system of the electronic device can employ a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture, and the like, which will not be described herein.

[0130] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be implemented independently or in combination. For the same or similar concepts or processes, some examples may not be described in detail.

[0131] The interface display method provided in the embodiments of the present application enables the voice assistant to pull up a dialogue window when detecting a user demand, and push services related to the user demand in the dialogue window.

[0132] 1. Pull up a dialogue window (Figures 3A-3G)

[0133] In response to the user performing a single-finger long press or a multi-finger long press operation in Figure 1B, the electronic device can display an interface as shown in Figure 3A. The multi-finger long press operation can include a double-finger long press operation or a triple-finger long press operation, and the like.

[0134] In Figure 3A, the note details interface, button 301, button 302, and dialogue window 303 can be displayed. The content displayed in the note details interface can be referred to the description in Figure 1B.

[0135] Button 301 can correspond to an intelligent cutout service, which can implement cutout of the note details interface. The word "intelligent cutout" can be displayed in button 301.

[0136] Button 302 can correspond to a same-item purchase service, which can search for any item in the note details interface. The word "same-item purchase" can be displayed in button 302.

[0137] Buttons 301 and 302 can be understood as service buttons. For example, in response to the user performing a single-finger long press or a multi-finger long press operation in Figure 1B, the electronic device can identify the note details interface in Figure 1B and push services related to the content in Figure 1B, i.e., the intelligent cutout service corresponding to button 301 and the same-item purchase service corresponding to button 302, and the like.

[0138] It can be understood that buttons 301 and 302 are only an example. In the case of displaying other content in the note details interface, the electronic device can also push service buttons different from buttons 301 and 302, which are not limited in the embodiments of the present application.

[0139] One or more of the following can be displayed in the dialog window 303: an identifier 304, a button 306, a button 311, a button 300, and a prompt information 3000.

[0140] The identifier 304 can be an identifier of a service corresponding to the voice assistant.

[0141] The button 306 can be used to adjust the size of the dialog window 303. For example, in response to an operation of the user pressing the button 306 and sliding upwards, the electronic device can enlarge the dialog window 303, such as enlarging the dialog window 303 to the size of the dialog window 305; or in response to an operation of the user pressing the button 306 and sliding downwards, the electronic device can reduce the dialog window 303 or close the dialog window 303.

[0142] The button 311 can be used to input other elements, such as pictures, videos, audios, or documents, etc., into the dialog window 303, in addition to text information. The process of inputting other elements into the dialog window based on the button 311 can be referred to the description in FIGS. 6B-6C.

[0143] The button 300 can be used to pull up a soft keyboard. For example, in response to an operation of the user clicking the button 300, the electronic device can display a soft keyboard as shown in FIG. 3A, so that the user can input text information into the dialog window 303 through the soft keyboard.

[0144] The prompt information 3000 can be used to prompt that the microphone is in an open state, for example, the prompt information 3000 can be displayed as the words “I am listening” or other words, which are not limited in the embodiments of the present application. As shown in FIG. 3A, when the prompt information 3000 is displayed in the dialog window 303, the prompt information 3000 indicates that the voice assistant is in a listening state. At this time, the microphone of the electronic device is in an open state.

[0145] Optionally, other information in addition to the prompt information 3000 can also be displayed in the dialog window 303 to prompt the user of the state of the voice assistant. For example, the words “thinking” can also be displayed in the dialog window 303 to prompt the user that the voice assistant is in a thinking state, etc., which are not limited in the embodiments of the present application.

[0146] It can be understood that the display state of the dialog window can include a folded state and an unfolded state, the folded state can be as shown in the dialog window 303, and the unfolded state can be as shown in the dialog window 305 in FIG. 3C. The dialog window can be switched between the folded state and the unfolded state, and the switching process can be referred to the description of the button 306.

[0147] In the case that the electronic device displays as shown in FIG. 3A, the electronic device can pull up the dialog window based on the operation of the user selecting a partial area in FIG. 3A and inputting voice information or text information of "where is this place" to the electronic device (see the description in Mode A). Alternatively, the electronic device can pull up the dialog window based on the operation of the user inputting voice information or text information of "where is the place in the picture" to the electronic device in FIG. 3A (see the description in Mode B).

[0148] Mode A: In response to the operation of the user selecting a partial area (e.g., the building area as shown in FIG. 3B) of the picture 103 in FIG. 3A and inputting voice information or text information of "where is this place" to the electronic device, the electronic device can display an interface as shown in FIG. 3C.

[0149] In an implementation of Mode A, when the electronic device displays the interface as shown in FIG. 3A, the microphone is turned on, and then the electronic device can acquire the voice information of "where is this place" input by the user through the microphone and detect the target operation of the user with respect to FIG. 3A, at which time the electronic device can display the interface as shown in FIG. 3B.

[0150] The target operation can include one or more of the following: an operation of single-finger circle selection of a partial area (e.g., the building area), or an operation of knuckle circle selection of a partial area (e.g., the building area), or an operation of double-finger stretching in the opposite direction to frame a partial area (e.g., the building area), and the like.

[0151] It can be understood that the electronic device can start the microphone to receive sound while displaying FIG. 3A, and the user can input demand 1 through voice, which can be "where is this place". The user can also supplement demand 2 through the target operation (e.g., the circle selection operation) in FIG. 3A, at which time the electronic device can display the circle selection track determined based on the target operation in FIG. 3B. After the voice operation and the circle selection operation are both completed, the electronic device can jump to the interface shown in FIG. 3C, and the voice assistant can push appropriate services in the dialog window based on demand 1 and demand 2. It can be understood that the order between the target operation and the voice operation is not limited in the embodiments of the present application.

[0152] In FIG. 3B, the note detail interface, the circle selection track 30, and the dialog window 303 can be displayed. The dialog window 303 is located on the note detail interface and blocks part of the note detail interface.

[0153] The circle selection track 30 can be a motion track formed when the user performs the above-mentioned circle selection operation. The circle selection track 30 can include the building area in the picture 103. The circle selection track 30 can be irregularly shaped, or the circle selection track 30 can also be regularly shaped, such as a rectangle, an ellipse, or a circle, and the like, which is not limited in the embodiments of the present application.

[0154] When the user ends both the target operation and the voice input operation in FIG. 3B, the electronic device can display the interface as shown in FIG. 3C.

[0155] It can be understood that there are three cases in the process in which the electronic device performs the target operation and the voice input operation in FIG. 3B. For example, the user ends the target operation first and then ends the voice input operation, or the user ends the voice input operation first and then ends the target operation, or the user ends both the target operation and the voice input operation at the same time.

[0156] In some embodiments, the user can perform the target operation and the voice input operation simultaneously or successively in FIG. 3B, and the target operation and the voice input operation can end at the same time. For example, after the target operation and the voice input operation end at the same time, the electronic device can continue to display the interface shown in FIG. 3B. When the electronic device does not detect other voice input operations within a period of time (such as 3 seconds, etc.), the electronic device can jump from FIG. 3B to the interface shown in FIG. 3C.

[0157] Alternatively, when the user ends both the target operation and the voice input operation at the same time, in response to the user's operation of pulling up the conversation window 303 in FIG. 3B, the electronic device can jump from FIG. 3B to the interface shown in FIG. 3C.

[0158] In some embodiments, the user can perform the target operation and the voice input operation simultaneously or successively in FIG. 3B, and end the voice input operation first and then end the target operation. For example, when the duration of the target operation is relatively long, so that the user ends the target operation after a period of time after ending the voice input operation, at this time the electronic device can continue to display FIG. 3B, and the electronic device can display the selection track 30 determined based on the target operation in FIG. 3B. When the electronic device does not detect other target voice input operations within a period of time (such as 3 seconds, etc.), the electronic device can jump from FIG. 3B to the interface shown in FIG. 3C.

[0159] Alternatively, when the user ends the target operation after a period of time after ending the voice input operation, at this time the electronic device can continue to display the interface shown in FIG. 3B. In response to the user's operation of pulling up the conversation window 303 in FIG. 3B, the electronic device can jump from FIG. 3B to the interface shown in FIG. 3C.

[0160] In some embodiments, the user can perform the target operation and the voice input operation simultaneously or successively in FIG. 3B, and end the target operation first and then end the voice input operation. For example, when the time length of the voice input operation is relatively long, so that the user ends the voice input operation after a period of time after ending the target operation, the electronic device can continue to display the interface shown in FIG. 3B, and the microphone is in a continuous sound receiving state. When the electronic device does not detect other voice input operations within a period of time (such as 3 seconds, etc.), the electronic device can jump from FIG. 3B to the interface shown in FIG. 3C.

[0161] Alternatively, when the user ends the voice input operation after a period of time after ending the target operation, in response to the user's operation of pulling up the dialogue window 303 in FIG. 3B, the electronic device can jump from FIG. 3B to the interface shown in FIG. 3C.

[0162] In possible implementations, there are also some abnormal situations in the process of the user performing the target operation and the voice input operation in FIG. 3B.

[0163] For example, in the process of the user performing the target operation and the voice input operation in FIG. 3B, ending the target operation, and not ending the voice input operation, in response to the user's operation of pulling up the dialogue window 303 in FIG. 3B, the electronic device continues to receive the user's voice input operation, at this time the electronic device can display the interface as shown in FIG. 3G. Further, when the user ends the voice input operation, the electronic device can display the interface shown in FIG. 3C.

[0164] Among them, the dialogue window 305 in FIG. 3G can display: the picture 320 generated based on the target operation, and the input box 321, which can display: prompt information for prompting the voice assistant to be in a listening state.

[0165] Alternatively, in the process of ending the target operation and not ending the voice input operation, in response to the user's operation of pulling up the dialogue window 303 in FIG. 3C, the electronic device can display the interface as shown in FIG. 3E. In this scenario, it can be understood that the voice input operation is interrupted.

[0166] Among them, the dialogue window 305 in FIG. 3E can display: the picture 320 generated based on the target operation. Optionally, the dialogue window 305 in FIG. 3E can also display: the message generated based on the user's voice input operation in FIG. 3B before the voice input operation is interrupted, such as the word "this place", which is not shown in FIG. 3E.

[0167] It can be understood that the processing process when the user only performs the voice input operation and the voice input operation is interrupted in FIG. 3B is similar to the above, which will not be described here.

[0168] It can be understood that the execution sequence between the target operation and the voice input operation is not limited in the embodiments of the present application.

[0169] As shown in FIG. 3C, the note detail interface and the dialogue window 305 overlaid on the note detail interface can be displayed. At this time, the voice assistant is in the thinking state.

[0170] It can be understood that the voice assistant can switch from the thinking state to the silent state after providing the reply message, or a period of time after providing the reply message, and no message is detected, etc., so as to save the device power consumption, and at this time, the microphone is in the closed state. For example, after the electronic device replies to the message 313 in FIG. 3C, or within 10 seconds after the electronic device displays FIG. 3C, no message is received, the voice assistant can switch from the thinking state to the silent state.

[0171] The top position of the dialogue window 305 can display the button 306, the button 307, the button 308, and the identifier 309, and the button 307 can be in the selected state.

[0172] The button 307 can be used to enter the first dialogue window of the voice assistant, that is, the dialogue window 305. The button 307 can display the word “YOYO Assistant” or the word of the voice assistant, etc.

[0173] It can be understood that a first dialogue window and a plurality of second dialogue windows can be provided in the voice assistant.

[0174] The button 308 can be used to select the intelligent agent. For example, in response to the click operation of the user on the button 308, the electronic device can display the interface as shown in FIG. 4C, which can include a plurality of intelligent agents.

[0175] The identifier 309 can be a pre-set identifier of the user, and the user's avatar can be displayed in the identifier 309.

[0176] The middle position of the dialogue window 305 can display the dialogue message, which can be composed of a plurality of message, including the user input message, the reply message provided by the electronic device based on the user input message. For example, in the case that the user input message includes the picture information in the circled track 30 and the text information “where is this place”, the reply message can include the text information “ABC store is located at XX street, XX, A city” and the message 313. The message 313 can indicate the position of the ABC store in the map application, and the message 313 can be a picture, a card or a link.

[0177] The bottom position of the dialogue window 305 can display the text box 310 and the button 312, and the button 311 can be displayed in the text box 310.

[0178] The text box 310 is used to input text information to the electronic device, and the text box 310 can display the text "please speak up".

[0179] The button 312 is used to turn on the microphone, and the button 312 can be displayed in the form of a microphone. For example, in response to the user clicking or long-pressing the button 312, the electronic device starts the microphone to collect audio. In response to the user inputting voice information 1, the electronic device can present the voice information 1 in the form of text or a card in the conversation window 305.

[0180] In this way, based on the description of FIGS. 3A-3C, the user can perform voice input while performing a target operation (for example, the above-mentioned circle selection operation). For example, the user can determine the area that needs to be inquired about through the target operation in FIG. 3B, and input a demand message for the area through voice input operation, so that the electronic device can provide a more accurate reply according to the content in the target operation and the voice input operation at the same time, so that the interaction mode is more rich and natural, and the efficient input of user demand is realized.

[0181] In another implementation of mode A, after the electronic device displays as shown in FIG. 3A, the electronic device can detect that the user performs a target operation for FIG. 3A, at this time, the microphone does not collect audio or the microphone is not turned on, and during the process of detecting the target operation by the electronic device, an interface as shown in FIG. 3D (the target operation is not ended) can be displayed.

[0182] After a period of time when the target operation ends, the electronic device can display an interface as shown in FIG. 3E. Alternatively, after the target operation ends, in response to the user pulling up the conversation window 303, the electronic device can display an interface as shown in FIG. 3E. When displaying the interface as shown in FIG. 3E, the voice assistant is in a thinking state, and at this time, the microphone can be in a closed state.

[0183] In response to the user performing a click operation on the button 312 in FIG. 3E, the voice assistant can be switched from the thinking state to the listening state, and the electronic device can start the microphone to collect audio.

[0184] The electronic device can collect the voice information "where is this place" input by the user through the microphone, and display an interface as shown in FIG. 3C after the audio collection ends. At this time, the voice assistant can be switched from the listening state to the thinking state, and the microphone is in a closed state.

[0185] Optionally, in the case of displaying the interface as shown in FIG. 3E, after the user inputs the text information "where is this place" in the text box 310, the electronic device can also display the interface as shown in FIG. 3C.

[0186] The conversation window 305 of FIG. 3E can include a picture 320.

[0187] The picture 320 can be a picture generated based on the user's circled part of the picture 103.

[0188] Optionally, the picture 320 can also be the same as the picture 103. Alternatively, the picture 320 determined based on the circled operation can also be displayed in the conversation window 305 at the same time as the picture 103, which is not shown in FIG. 3E, and the present embodiments do not limit the same.

[0189] Optionally, the conversation window 305 in FIG. 3E can also include a reply message provided by the voice assistant based on the picture 320, such as a reply message displayed as "whether to search the picture" or "please ask questions related to the picture" and the like, which is not shown in FIG. 3E. It can be understood that in the case where the electronic device provides a reply message based on the picture 320, the electronic device can also provide a more accurate reply based on the picture 320 and the voice information in response to the user's operation of continuing to input voice information through the button 312.

[0190] In this way, based on the descriptions of FIGS. 3A, 3D, 3E and 3C, the electronic device can support the user to supplement other needs for the selected region after performing the target operation. In this scenario, the user can focus on the target operation and the need supplement, respectively.

[0191] Method B: in response to the user inputting the voice information or text information of "where is the location in the picture" to the electronic device in FIG. 3A, the electronic device can identify the location in the picture 103, and then display the identification result in FIG. 3F.

[0192] In one implementation of method B, after the electronic device displays as shown in FIG. 3A, the electronic device can collect the voice information of "where is the location in the picture" input by the user through the microphone. Further, after a period of time after the voice input operation ends, the electronic device can display the interface shown in FIG. 3F; or, in response to the user's operation of pulling up the conversation window 303 after the voice input operation ends, the electronic device can display the interface as shown in FIG. 3F.

[0193] In the process of displaying FIG. 3F, the electronic device can first be in a thinking state, and then display the reply message related to both the message 316 and the message 317 in sequence after the thinking is completed. When the electronic device has displayed the reply message related to both the message 316 and the message 317 in FIG. 3F, the voice assistant can be switched from the thinking state to the silent state.

[0194] That is, compared with method A, method B can simplify the steps of determining the user's needs based on the circled operation, and provide more diversified interaction modes.

[0195] The dialog window 305 of FIG. 3F can display: the message 316, the message 317, and a reply message generated by the electronic device based on the message 316 and the message 317. The message 316 can display: a thumbnail corresponding to the picture 103; and the message 317 can display: the text "where is the location in the picture".

[0196] It can be understood that the message 316 can also only include a location in the picture 103, such as a building region in the picture 103. Alternatively, the message 316 can also display a picture, a screenshot, or the like generated based on the note detail interface. The present application does not limit the order of the message 316 and the message 317, and the content included in the message 316 or the message 317.

[0197] In another implementation of the manner B, after the electronic device displays the interface as shown in FIG. 3A, in response to the operation of the user inputting the text information "where is the location in the picture" through the button 300, the electronic device can display the interface as shown in FIG. 3F.

[0198] In the description of the manner B, the user can clearly describe the user demand through voice input or text input only, so as to achieve the purpose of simplifying the use of the voice assistant and improving the user demand.

[0199] 2. Push different services in the dialog window (FIGS. 4A-4K)

[0200] In the embodiments corresponding to FIGS. 4A-4K, the electronic device can provide a reply message related to the message A through the service A (or called intelligent agent A) when detecting the message A input by the user, or the electronic device can provide a reply message related to the message B through the service B (or called intelligent agent B) when detecting the message B input by the user.

[0201] It can be understood that, in response to the operation of the user inputting the message A, in a case where the electronic device determines that the voice assistant cannot provide a reply message related to the message A, the electronic device can distribute the message A to the service A, so that the service A can provide a more accurate reply message. Alternatively, in response to the operation of the user inputting the message B, in a case where the electronic device determines that the voice assistant cannot provide a reply message related to the message B, the electronic device can distribute the message B to the service B, so that the service B can provide a more accurate reply message.

[0202] In the embodiments corresponding to FIGS. 4A-4F, the message A is taken as an air ticket reservation message, and the service A is taken as an intelligent travel assistant as an example for illustration, which does not constitute a limitation on the embodiments of the present application.

[0203] It can be understood that the electronic device can provide the reply message corresponding to the message A through the service A in the first conversation window (see the description in mode I); or the electronic device can also provide the reply message corresponding to the message A through the service A in the second conversation window (such as the conversation window corresponding to the service A) (see mode II).

[0204] Mode I: In response to the operation of the user inputting the message A of "Help me book a flight to next Monday" in FIG. 3C, the electronic device displays the interface as shown in FIG. 4A. Further, the electronic device can identify the message A and provide the corresponding reply message through the smart travel assistant (i.e., the service A) and display the interface as shown in FIG. 4B.

[0205] Optionally, the message A can also be a message generated by the user based on the voice input operation in FIG. 3A.

[0206] The message A can be displayed in the conversation window 305 of FIG. 4A. The message A can be a flight booking message, for example, the message A can be displayed as "Help me book a flight to next Monday". Other contents displayed in FIG. 4A can be similar to those in FIG. 3C, which will not be described here.

[0207] The reply message related to the message A, such as the message 404 and the message 405, can be displayed in the conversation window 305 of FIG. 4B. For example, the message 404 can indicate that the electronic device has searched for flights to A city, and the message 405 can indicate the specific information of the flight. As shown in the message 405, the flight information from B city T1 airport to A city T2 airport from 07:30 to 09:00, and the flight information from B city T2 airport to A city T2 airport from 10:00 to 12:00, etc. can be displayed. Wherein, B city can be understood as the city where the user is located, and A city can be understood as the city where ABC store (exhibition center store) is located.

[0208] The message 404 and the message 405 can be messages provided by the smart travel assistant. In this scenario, the upper side of the message 404 can also display the content related to the smart travel assistant. For example, one or more of the following can be displayed: the service identifier 401, the service name 402, or the prompt information 403.

[0209] The service identifier 401 can be displayed as an icon of the smart travel assistant, and the style of the icon is only illustrative; the service name 402 can be displayed as "smart travel assistant"; and the prompt information 403 can be used to prompt that the relevant service is being provided for the user, for example, the prompt information 403 can be displayed as "serving you".

[0210] Optionally, the content displayed in the prompt information 403 can also be changed according to the service provided by the intelligent agent, which is not limited in the embodiments of the present application.

[0211] It can be understood that, in a case where the electronic device determines that the corresponding reply message can be provided by the smart travel assistant based on the message A, the user can be prompted that the message A has been distributed to the smart travel assistant by displaying the service identifier 401 and / or the service name 402 and the like, so that the smart travel assistant can provide more accurate replies to improve the user's use experience.

[0212] Based on the description of FIGS. 4A-4B, the electronic device can determine whether the demand message of the user needs to be distributed to a certain agent in the voice assistant when detecting the demand message. When the electronic device determines that the demand message needs to be distributed to a certain agent, the demand message can be distributed to the agent, and then the agent can provide a reply message corresponding to the demand message. Or, when the electronic device determines that the demand message does not need to be distributed to the agent, the voice assistant can provide a reply message corresponding to the demand message in combination with user personal habit data and the like, or the voice assistant can also provide a reply message corresponding to the demand message through objective third-party data.

[0213] For example, when the demand message of the user is a query weather message, the electronic device can determine that the demand message does not need to be distributed to the agent, and at this time the voice assistant can realize the query of the current weather through objective weather data.

[0214] For example, when the demand message of the user is a message for reminding the schedule on the business trip day, the electronic device can determine that the demand message does not need to be distributed to the agent, and at this time the voice assistant can provide a reply message through user personal habit data, such as querying the business trip day in the schedule service and then setting a schedule reminder on the business trip day.

[0215] For example, the demand message of the user is a flight subscription message, the electronic device can determine that the demand message needs to be distributed to the agent (such as the smart travel assistant), and then the electronic device distributes the demand message to the smart travel assistant, and the smart travel assistant can provide a reply message related to the flight subscription message.

[0216] In this way, the electronic device distributes the service in the mode I, so that the user can not only view the reply message provided by the service A in the first dialogue window, but also view all the dialogue messages in the first dialogue window at any time.

[0217] Mode II: In response to the user performing a click operation on the button 308 in FIG. 3C, the electronic device displays an interface as shown in FIG. 4C. In response to the user performing a click operation on the button 409 in FIG. 4C, the electronic device displays an interface as shown in FIG. 4D.

[0218] The window 406 in FIG. 4C can be used to display at least one intelligent agent. For example, the window 406 can display: an option bar 407, and at least one intelligent agent under a recent option button 408 in the option bar 407.

[0219] The option bar 407 can include one or more of the following: the recent option button 408, a recommended option button, a travel option button, a sports option button, and the like.

[0220] The recent option button 408 can be used to view intelligent agents that have been used by the user in a period of time or that are set by default by the electronic device; the recommended option button can be used to display intelligent agents recommended by the electronic device; the travel option button can be used to view intelligent agents related to a travel scenario; and the sports option button can be used to view intelligent agents related to a sports and health scenario.

[0221] When the recent option button 408 is in a selected state, the at least one intelligent agent under the recent option button 408 can include one or more of the following: a button 409 corresponding to a smart travel assistant, a button corresponding to a smart photo album assistant, or a button 410 corresponding to a life service assistant.

[0222] The button 409 can display: a service identifier of the smart travel assistant, a service name of the smart travel assistant, and a service introduction of the smart travel assistant, and the like. For example, the service introduction of the smart travel assistant can include: providing ticket checking services, travel services, and the like.

[0223] The button corresponding to the smart photo album assistant can display: a service identifier of the smart photo album assistant, a service name of the smart photo album assistant, and a service introduction of the smart photo album assistant, and the like. For example, the service introduction of the smart photo album assistant can include: automatically generating a video in a period of time for the user, and the like.

[0224] The button 410 can display: a service identifier of the life service assistant, a service name of the life service assistant, and a service introduction of the life service assistant, and the like. For example, the service introduction of the life service assistant can include: a one-stop life concierge.

[0225] It can be understood that the intelligent agents displayed in the window 406 in FIG. 4C are only illustrative and do not constitute a limitation on the embodiments of the present application.

[0226] The window 406 in FIG. 4C can also display: the button 306, the button 307, and the button 308, and the button 308 can be in a selected state. The meanings of the button 306, the button 307, and the button 308 are not repeated here. For example, in response to the user performing an operation on the button 307 in FIG. 4C, the electronic device can display the interface shown in FIG. 3C.

[0227] The dialog window 411 in FIG. 4D can include a dialog window corresponding to the intelligent travel assistant, which can also be referred to as a second dialog window. The intelligent travel assistant can be service A.

[0228] The dialog window 411 can display the service name of the intelligent travel assistant, the service identifier of the intelligent travel assistant, the slogan 412 of the intelligent travel assistant (such as "experience intelligent travel"), the button 417, and the message 413.

[0229] The button 417 can be used to exit the dialog window 411. For example, in response to the user performing an operation on the button 417 in FIG. 4D, the electronic device can display the interface shown in FIG. 4C.

[0230] The message 413 can be a message displayed by default when the intelligent travel assistant is opened. The message 413 can be used to prompt the user to enter the intelligent travel assistant.

[0231] Optionally, the message 413 can not be displayed in the dialog window 411, or other content can be displayed in the dialog window 411 in addition to the message 413, such as a historical message in the intelligent travel assistant, and the like, which are not limited in the embodiments of the present application.

[0232] The bottom position of the dialog window 411 can display the text box 433, and the button 434 can be displayed in the text box 433. The meaning of the button 434 can be referred to the description of the button 311 in FIG. 3C, and the meaning of the button 435 can be referred to the description of the button 312 in FIG. 3C, which will not be repeated here.

[0233] In response to the user inputting the voice information or text information "help me book a ticket to A city next week" in FIG. 4D, the electronic device can display the interface shown in FIG. 4E and the interface shown in FIG. 4F in sequence.

[0234] It can be understood that since the dialog window 305 is a first dialog window and the dialog window 411 is a second dialog window, the message in the first dialog window cannot be translated into the second dialog window, so the user needs to re-input the user demand (such as the message 414) after opening the dialog window 411, so that the service in the second dialog window can reply to the message.

[0235] The message 414 input by the user can be displayed in the dialog window 411 of FIG. 4E, and the message 414 can be message A.

[0236] The reply message provided by the electronic device based on the message 414 can be displayed in the dialogue window 411 of FIG. 4F, such as the message 415 which can be the same as the message 404, and the message 416 which can be the same as the message 405, and details are not described herein again.

[0237] In this way, the electronic device provides the service push in the mode II, so that the user can select a suitable agent according to the demand, and input the demand in the second dialogue window (i.e., the dialogue window 411) to provide a more accurate reply message by the electronic device.

[0238] In the embodiments corresponding to FIGS. 4G-4K, the message B is taken as the restaurant query message, and the service B is taken as the life service agent for example, and the example does not constitute a limitation on the embodiments of the present application.

[0239] It can be understood that the electronic device can provide the reply message corresponding to the message B by the service B in the first dialogue window (see the description in mode III); or the electronic device can also provide the reply message corresponding to the message B by the service B in the second dialogue window (such as the dialogue window corresponding to the service B) (see mode IV).

[0240] Mode III: In response to the operation of the user inputting the information B of "how is the taste of this restaurant" in FIG. 3C, the electronic device displays the interface shown in FIG. 4G. Then the electronic device can identify the message B, and provide the corresponding reply message by the life service agent (i.e., the service B) and display the interface shown in FIG. 4H.

[0241] The dialogue window 305 of FIG. 4G can display the message B, which can be a restaurant query message, such as the message B displayed as "how is the taste of this restaurant". The other contents displayed in FIG. 4G can be similar to those in FIG. 3C, and details are not described herein again.

[0242] The dialogue window 305 of FIG. 4H can display the reply message related to the message B, such as the message 424 and the message 425. For example, the message 424 can indicate that the electronic device has queried the store information, and the message 425 can indicate the specific situation of the store information, such as the content displayed in the message 425 that the ABC store (exhibition center store) has a score of 5 points, and the ABC store (exhibition center store) is a dessert store located near the exhibition center.

[0243] The message 424 and the message 425 can be the messages provided by the life service agent, and therefore the upper side of the message 424 can also display the content related to the life service agent, such as the service identifier 421, the service name 422, and the prompt information 423.

[0244] The service identifier 401 can be displayed as an icon of the life service assistant, and the style of the icon is only illustrative. The service name 422 can be displayed as the word "life service assistant". The prompt information 423 is the same as the prompt information 403, and is not described herein again.

[0245] Optionally, when the message 313 in FIG. 3C and the text information "ABC store is located at XX street XX, A city" are messages provided by the life service assistant, the service identifier 421, the service name 422, and the prompt information 423 can also be displayed around the message 313, which is not shown in FIG. 3C.

[0246] It can be understood that, in a case where the electronic device determines that the corresponding reply message can be provided by the life service assistant based on the message B, the user can be prompted that the current service has been switched to a more suitable service by displaying the service identifier 421 and / or the service name 422 and the like, so as to improve the user's use experience.

[0247] In this way, the electronic device can not only view the reply message provided by the service B in the first dialogue window, but also view all the dialogue messages in the first dialogue window at any time by the service pushing manner III.

[0248] IV: In response to the user performing a click operation on the button 308 in FIG. 3C, the electronic device displays an interface as shown in FIG. 4C. In response to the user performing a click operation on the button 410 in FIG. 4C, the electronic device displays an interface as shown in FIG. 4I.

[0249] The dialogue window 428 can be included in FIG. 4I, and the dialogue window 428 can be a dialogue window corresponding to the life service assistant, and the dialogue window 428 can also be referred to as a second dialogue window.

[0250] The service name of the life service assistant, the service identifier of the life service assistant, the slogan 440 of the life service assistant (such as life entertainment and food little helper), the button 427, the message 429, and the button 434 can be displayed in the dialogue window 428. The life service assistant can be the service B.

[0251] The button 427 can be used to exit the dialogue window 428. For example, in response to the user performing an operation on the button 427 in FIG. 4I, the electronic device can display an interface as shown in FIG. 4C.

[0252] The message 429 can be a message displayed by default when the life service assistant is opened. The message 429 can be used to prompt the entry of the life service assistant.

[0253] Optionally, in response to the user inputting the text message of "where is this place" in FIG. 4I and the operation of displaying the building area in FIG. 3B, the electronic device can also display the text message of "ABC store is located at XX street, XX, A city" in FIG. 4I and / or the picture message of the address of the ABC store in FIG. 3C.

[0254] Optionally, the message 429 can not be displayed in the dialogue window 428, or other content can be displayed in the dialogue window 428 in addition to the message 429, such as historical messages in the life service assistant, and the like, which are not limited in the embodiments of the present application.

[0255] In response to the user inputting the voice information or the text information of "help me query the score of ABC store (exhibition center store)" in FIG. 4I, the electronic device can display the interface shown in FIG. 4J and the interface shown in FIG. 4K in sequence.

[0256] The message 430 input by the user can be displayed in the dialogue window 428 in FIG. 4J, and the message 430 can be message B.

[0257] It can be understood that the message 430 can be different from the content displayed in the message B in FIG. 4G. Or it can be understood that, in the case that the store information has been displayed in the dialogue window 305 in FIG. 4G, the user can inquire about the store by inputting the message B; and in the case that the store information is not displayed in the dialogue window 428 in FIG. 4J, the user can query the store information by inputting the message 430, and the specific content displayed in the message B is not limited in the embodiments of the present application.

[0258] The reply message provided by the electronic device based on the message 430 can be displayed in the dialogue window 428 in FIG. 4K, such as the message 431 and the message 432. The message 431 can be the same as the message 424, and the message 432 can be the same as the message 425, which are not described herein again.

[0259] In this way, the electronic device pushes the service by the mode IV, so that the user can select a suitable intelligent agent according to the demand, and input the demand in the second dialogue window (i.e., the dialogue window 428) to provide more accurate reply messages by the electronic device.

[0260] In combination with the description of manners I-IV, when the user inputs different messages in the first dialogue window, the voice assistant can provide different reply messages corresponding to the different messages respectively through different intelligent agents. For example, when the user inputs message A in the first dialogue window, the electronic device can provide a reply message related to message A in the first dialogue window through service A; or when the user inputs message B in the first dialogue window, the electronic device can provide a reply message related to message B in the first dialogue window through service B. In this way, the user can view the reply message provided by any sub-service in the voice assistant without interface switching, achieving the purpose of simplifying user use.

[0261] 3. Changing user needs to update the content in the dialogue window (Figures 5A-5D)

[0262] In response to the user performing the operation of pressing the button 306 and sliding downward in Figure 3C, the electronic device can display the interface as shown in Figure 5A.

[0263] In Figure 5A, the note details interface and the dialogue window 303 can be displayed. The content contained in the dialogue window 303 can be referred to the description of Figure 3A, which will not be repeated here.

[0264] It can be understood that in response to the user performing the operation of pressing the button 306 and sliding downward in Figure 3C, the electronic device can start the microphone and display the interface as shown in Figure 5B. At this time, the dialogue window 303 can display the words "I am listening" to prompt the user that the microphone is in an open state. In this way, the user can input messages through the microphone.

[0265] Alternatively, in response to the user performing the operation of pressing the button 306 and sliding downward in Figure 3C, the electronic device can keep the microphone in a closed state. At this time, the dialogue window 303 in Figure 5A can not display the words "I am listening". At this time, the user can input text information through the button 300 in the dialogue window 303, or start the microphone through other operations, which are not limited in the embodiments of the present application.

[0266] In response to the user performing the operation of selecting a part of the area of the picture 103 (such as the area where the teacup is located) in Figure 3A and inputting voice information such as "where can I buy this" to the electronic device, the electronic device can display the interface as shown in Figure 5B (the selection operation is not ended) at this time. After a period of time when the selection operation is ended, the electronic device can display the interface as shown in Figure 5C and the interface as shown in Figure 5D in turn.

[0267] In Figure 5B, the note details interface, the selection track 50, and the dialogue window 303 can be displayed.

[0268] The circled track 50 can include a region in which the cup in the picture 103 is located. The form of the circled track 50 can refer to the description of the circled track 30, which is not repeated here.

[0269] The dialog window 305 can be displayed in FIG. 5C. The dialog window 305 can display the message 502 and the message 503. The message 502 can be a picture generated based on the user's circled operation in FIG. 5B, and the message 503 can be a message generated based on the voice information input by the user in FIG. 5B, which can be displayed as "where to buy".

[0270] Optionally, in response to the user performing the circled operation on the region of the picture 103 (such as the region in which the cup is located) in FIG. 3A, the electronic device can also display only the message 502 in the dialog window 305 in FIG. 5C, and display the message 504 and / or the message 505 in the dialog window 305 in FIG. 5D, which is not limited in the embodiments of the present application.

[0271] It can be understood that the dialog window 305 in FIG. 5C can also display historical messages such as the message 313, and the content of the message 313 can refer to the description of FIG. 3C, which is not repeated here.

[0272] The dialog window 305 in FIG. 5D can display reply messages related to the message 502 and the message 503, such as the message 504 and the message 505. For example, the message 504 can indicate that the electronic device has searched for similar goods, and the message 505 can display a plurality of searched goods.

[0273] The message 504 and the message 505 can be messages provided by a life service assistant, and therefore the upper side of the message 504 can also display content related to the life service assistant. For example, the service identifier of the life service assistant, the service name of the life service assistant, and the prompt information displayed as "serving you" can be displayed.

[0274] Optionally, the user can also perform the operation of circling the part of the picture 103 and inputting the voice information or text information of "where to buy" based on other implementations described in the above manner A, and the specific process is not repeated here.

[0275] In this way, based on the description in FIGS. 5A-5D, the electronic device can not only support the user to change the selected region, but also provide related service content based on the changed selected region of the user, so as to improve the user experience of using the voice assistant. It can be understood that during the process of changing the selected region by the user, the dialog window can be in a retracted state to reduce the coverage of the interface content.

[0276] It can be understood that, in response to the user's sliding operation on the picture in FIG. 5A, the electronic device can display other pictures in the note, and the electronic device can support the user to change the selected area in the other pictures. Alternatively, in response to the user's upward sliding operation or downward sliding operation on the picture in FIG. 5A, the electronic device can display other content in the note, such as other information in the note, or comment information related to the note, and the electronic device can also support the user to change the selected area in the other content of the note, which is not limited in the embodiments of the present application.

[0277] Optionally, the operation of changing the selected area can not be limited to the user's circle selection operation on the partial area of the picture, or can be the circle selection operation on the text information in the note, or the selection operation on the text information in the note, which is not limited in the embodiments of the present application.

[0278] 4. Inserting other elements in the dialogue window (FIGS. 6A-6F)

[0279] The other elements can include one or more of the following: pictures, videos, audios, or documents, etc. The documents can include one or more of the following: picture files, video files, audio files, compressed packages, or web page files, etc.

[0280] In response to the user's operation of dragging the text information in the dashed box 102 in FIG. 1B to the edge of the screen, the electronic device can display an interface as shown in FIG. 6A, which can display the picture 104, the thumbnail 6000, and the service list 600.

[0281] The thumbnail 6000 can display the text information in the dashed box 102, such as "ABC Store (Exhibition Center Store)".

[0282] The service list 600 can display buttons of multiple services. For example, the service list 600 can display a button 601, and other buttons displayed in the service list 600 can refer to the description of the service list 105. Optionally, the button 601 can be arranged at the top of the service list 600, so that the user can quickly find the voice assistant corresponding to the button 601 when the service list 600 is pulled up.

[0283] The button 601 can display the service identifier of the voice assistant and the service name of the voice assistant, and the button 601 can be used to open the first dialogue window in the voice assistant.

[0284] In response to the user's click operation on the button 601 in FIG. 6A, the electronic device displays an interface as shown in FIG. 6B.

[0285] The interface in FIG. 6B can display a note detail interface and a dialogue window 606.

[0286] The message 60 and the message 620 can be displayed in the conversation window 606. The message 60 can be used to prompt entering the voice assistant, and the message 60 can be displayed as the words "Come and talk to me". The message 620 can be a message generated based on the text information in the thumbnail 6000.

[0287] Optionally, the message 621 can also be displayed in the conversation window 606, and the message 621 can be a reply message provided by the voice assistant based on the message 620. The message 621 can be used to prompt querying the address of the message 620.

[0288] Optionally, no message can be displayed in the conversation window 606, or other messages can be displayed in the conversation window 606 in addition to the message 60, and the embodiments of the present application do not limit this.

[0289] The button 602 can also be displayed in the conversation window 606, and the button 602 can be used to input other elements in addition to the text information into the conversation window.

[0290] In response to the user performing a click operation on the button 602 in FIG. 6B, the electronic device displays an interface as shown in FIG. 6C. The button 603, the button 604 and the button 605 can be displayed in the conversation window 606 in FIG. 6C.

[0291] The button 603 can be used to start the camera to take a picture. For example, in response to the user's operation on the button 603, the electronic device can display a camera interface, and in response to the user's operation on a shooting button in the camera interface, the electronic device can take a picture or a video, and then input the picture or the video into the conversation window 606.

[0292] The button 604 can be used to open a gallery application.

[0293] The button 605 can be used to open a document application. In response to the user's operation on the button 605, the electronic device can display a document interface, and in response to the user's operation of selecting a document 1 in the document interface, the electronic device can input the document 1 into the conversation window 606. The document interface can be an interface in the document application, and the document application can also be referred to as a file management application, or a file manager, etc. The document application can be used to store all documents in the electronic device.

[0294] The following takes an example of inputting a picture into the conversation window 606 by triggering the button 604, which does not constitute a limitation on the embodiments of the present application.

[0295] In response to the user performing a click operation on the button 604 in FIG. 6C, the electronic device displays an interface as shown in FIG. 6D.

[0296] As shown in FIG. 6D, the electronic device can display a thumbnail of at least one picture in the gallery application, and a control button 607 at the bottom. The gallery application can also be referred to as a photo album application, and the gallery application can store pictures, videos, and other content in the electronic device.

[0297] For example, as shown in FIG. 6D, the thumbnail 608 can be included, and the upper right corner of the thumbnail 608 can display a check button 609.

[0298] The control button 607 can display, from left to right, a preview button, a button for checking the original picture, and a send button 612.

[0299] In response to the user checking the thumbnail 608 (e.g., clicking the check button 609) in FIG. 6D and then clicking the send button 612, the electronic device displays an interface as shown in FIG. 6E.

[0300] The message 613 can be displayed in the conversation window 606 of FIG. 6E, and the message 613 can be a message generated based on the picture in the thumbnail 608.

[0301] In response to the user inputting the message 610 in FIG. 6E, the electronic device can display an interface as shown in FIG. 6F.

[0302] The message 610, the message 611, and the message 615 can be displayed in the conversation window 606 of FIG. 6F. The message 611 and the message 615 can be reply messages provided by the life service assistant in the voice assistant based on the message 610 and the message 613. For example, the message 615 can be used to prompt the user to inquire about the dishes in the ABC store (exhibition center store), and the message 611 can display the details of the ABC store (exhibition center store).

[0303] The message 611 and the message 615 can be messages provided by the life service assistant in the voice assistant, and the upper side of the message 611 can also display the service identifier and the service name of the life service assistant, and the like, which are not limited in the embodiments of the present application.

[0304] It can be understood that when the user enters any intelligent agent, the electronic device can also input other elements into the conversation window corresponding to the intelligent agent based on the buttons in the conversation window corresponding to the intelligent agent. For example, the user can input the message 613 into the conversation window 428 by clicking the button 434 in FIG. 41, and the specific process is similar to that described in FIGS. 6C-6F, which will not be described here.

[0305] It can be understood that, in a case where the user inputs a photographed picture or video to the conversation window 606 through the button 603, the voice assistant can also provide a corresponding reply message according to the photographed picture or video; or, in a case where the user inputs a document to the conversation window through the button 605, the voice assistant can also provide a corresponding reply message based on the content of the document, and details are not described herein again.

[0306] In this way, the electronic device can pull up the conversation window 606 by dragging the selected content in FIG. 5A to the position where the button 601 in the service list 600 is located, and send the selected content to the conversation window 606, so that the voice assistant can provide a reply related to the selected content in FIG. 5A.

[0307] The electronic device sets the button 601 corresponding to the voice assistant in the service list 600 in such a manner that the user can quickly start the voice assistant and ask about the display content in the interface in the voice assistant. In addition, the voice assistant can support input of multiple types of content, so as to provide more accurate services for the user.

[0308] In combination with the descriptions in FIGS. 3A-6F, FIG. 7 is a flow diagram of an interface display method provided in an embodiment of the present application. As shown in FIG. 7, the interface display method can include the following steps:

[0309] S701, the electronic device displays a first interface.

[0310] The first interface includes a first element, and the first element includes one or more of the following: text, a picture, a video, or a file.

[0311] The first interface can be the interface shown in FIG. 1B, and the first element can be the element in FIG. 1B.

[0312] S702, after receiving a first operation on the first interface, the electronic device displays a second interface.

[0313] The second interface includes the display content of the first interface, an identifier of a first voice assistant, and first prompt information, and the first prompt information is used to prompt that the microphone is receiving sound.

[0314] The first operation can be a single-finger long press or a multi-finger long press performed by the user in FIG. 1B.

[0315] The second interface can be the interface shown in FIG. 3B. As shown in FIG. 3B, the identifier of the first voice assistant can be the identifier 304, and the first prompt information can be the prompt information 3000.

[0316] The first voice assistant can be the voice assistant described in the embodiments of the present application.

[0317] S703. After receiving the first message acquired through the microphone, the electronic device displays a third interface.

[0318] The third interface includes a first window, and the first window includes a second message, a third message, and a fourth message. The first message is a message related to the first element. The second message includes text corresponding to the first message. The third message includes a picture corresponding to the first element. The fourth message is a reply message provided based on the second message and the third message.

[0319] In an implementation, the first message can be a voice message, and the third interface can be the interface shown in FIG. 3C. As shown in FIG. 3C, the first window can be a conversation window 305, the second message can be a text message generated based on the voice message, such as the text message "where is this place", the third message can be the building area shown in FIG. 3B, and the fourth message can include the text information "ABC store is located at XX Street XX, A City" and / or the message 313.

[0320] In another implementation, the first message can be a voice message for selecting the first element in FIG. 3A, and the third interface can be the interface shown in FIG. 3F. As shown in FIG. 3F, the first window can be a conversation window 305, the second message can be a text message generated based on the voice message, such as the message 317, the third message can be the message 316, and the fourth message can include the text information "ABC store is located at XX Street XX, A City" and / or the picture message showing the address of the ABC store.

[0321] Based on this, the electronic device can conveniently push the message related to the first message and the third message (such as the related picture in the interface) based on the operation of the user inputting the first message by voice, so that the user can provide a more accurate reply message based on the voice input and in combination with the content in the current interface while performing voice interaction with the voice assistant, thereby improving the user experience.

[0322] It should be noted that the interface provided by the embodiments of the present application and the content that can be displayed in the interface are only used as an example and do not constitute a limitation on the embodiments of the present application.

[0323] It should be noted that the module names involved in the embodiments of the present application can be defined as other names, as long as the functions of the modules can be implemented, and the names of the modules are not limited specifically.

[0324] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0325] The interface display method of the embodiments of the present application has been described above, and the device for executing the above method provided by the embodiments of the present application will be described below. Those skilled in the art can understand that the method and the device can be combined and referred to each other, and the related device provided by the embodiments of the present application can execute the steps in the list sorting method.

[0326] FIG. 8 is a schematic diagram of the hardware structure of another electronic device provided by the embodiments of the present application.

[0327] The electronic device includes a processor 801, a communication line 804 and at least one communication interface (for example, the communication interface 803 is exemplarily illustrated in FIG. 8).

[0328] The processor 801 can be a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the present application scheme.

[0329] The communication line 804 can include a circuit for transmitting information between the above-mentioned components.

[0330] The communication interface 803 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (WLAN), etc.

[0331] Possibly, the electronic device can further include a memory 802.

[0332] The memory 802 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory can exist independently and be connected to the processor through the communication line 804. The memory can also be integrated with the processor.

[0333] The memory 802 is configured to store computer-executable instructions for implementing the solutions of the present application, and the processor 801 is configured to execute the computer-executable instructions stored in the memory 802.

[0334] Possibly, the computer-executable instructions in the embodiments of the present application can also be referred to as application program codes, which are not specifically limited in the embodiments of the present application.

[0335] In a specific implementation, as an embodiment, the processor 801 can include one or more CPUs, such as CPU0 and CPU1 in FIG. 8.

[0336] In a specific implementation, as an embodiment, the electronic device can include multiple processors, such as the processor 801 and the processor 805 in FIG. 8. Each of the processors can be a single-CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (for example, computer program instructions).

[0337] The interface display method provided by the embodiments of the present application can be applied to an electronic device with a communication function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be described here.

[0338] The embodiment of the present application provides a terminal device, which comprises a processor and a memory; the memory stores computer-executed instructions; and the processor executes the computer-executed instructions stored in the memory, so that the terminal device executes the method.

[0339] The embodiment of the present application provides a chip. The chip comprises a processor configured to invoke a computer program in a memory to execute the technical solutions in the above-described embodiments. The implementation principle and technical effects are similar to those of the above-described related embodiments, which will not be described herein again.

[0340] The embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program is executed by a processor to implement the above-described method. The method described in the above-described embodiments can be implemented by software, hardware, firmware or any combination thereof in whole or in part. If implemented in software, the functions can be stored in or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium can include a computer storage medium and a communication medium, and can further include any medium that can carry computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0341] In a possible implementation, the computer-readable medium can include a RAM, a ROM, a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage device, or any other medium that is targeted to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, the optical fiber cable, the twisted pair, the DSL or the wireless technology (such as infrared, radio and microwave) is included in the definition of the medium. As used herein, a disk and a disc include a compact disc, a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disk and a Blu-ray disc, wherein the disk usually magnetically reproduces data, and the disc optically reproduces data with a laser. The above combinations should also be included in the scope of the computer-readable medium.

[0342] The embodiment of the present application provides a computer program product, which comprises a computer program. When the computer program is executed, the computer program causes a computer to execute the above-described method.

[0343] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0344] The above detailed description has further explained the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. A method for displaying an interface, characterized in that, The methods include: Display a first interface, the first interface including a first element, the first element including one or more of the following: text, image, video or file; Upon receiving a first operation on the first interface, a second interface is displayed, which includes: the content displayed on the first interface, the identifier of the first voice assistant, and a first prompt message, which is used to indicate that the microphone is picking up sound. Upon receiving the first message acquired through the microphone, a third interface is displayed, comprising: a first window, which includes: a second message, a third message, and a fourth message. The first message is a message related to the first element, the second message includes text corresponding to the first message, the third message includes an image corresponding to the first element, and the fourth message is a response message based on the second message and the third message.

2. The method according to claim 1, characterized in that, Following the display of the second interface, the following is also included: The system receives a second operation, which is an operation on a first element in the second interface, and the third message is an image generated based on the second operation.

3. The method according to claim 2, characterized in that, Before displaying the third interface, the method further includes: During the process of receiving the second operation, the motion trajectory of the second operation is displayed on the second interface; When the first message ends at the first moment and the second operation has not ended, the motion trajectory of the second operation continues to be displayed on the second interface. The third interface is the interface displayed after the second operation ends at the second moment, and the first moment is earlier than the second moment.

4. The method according to claim 2, characterized in that, Before displaying the third interface, the method further includes: During the process of receiving the second operation, the motion trajectory of the second operation is displayed on the second interface; When the second operation ends at the third moment and the first message has not ended, the first message continues to be acquired through the microphone. The third interface is the interface displayed after the first message ends at the fourth moment, and the third moment is earlier than the fourth moment.

5. The method according to claim 4, characterized in that, The second interface also includes: a first button, and when the second operation ends at the third time and the first message has not ended, it also includes: Upon receiving an operation on the first button, a fourth interface is displayed, wherein the fourth interface includes a second window, the second window including the third message and the first prompt information, or the second window includes the third message and a fifth message, the fifth message being a text message generated based on the unfinished first message.

6. The method according to claim 2, characterized in that, Before displaying the third interface, it also includes: During the process of receiving the second operation, the motion trajectory of the second operation is displayed on the second interface, and the third interface is the interface displayed when the second operation and the first message end simultaneously.

7. The method according to any one of claims 2-6, characterized in that, The second interface also includes: a first button, Before displaying the third interface, the method further includes: displaying the motion trajectory of the second operation in the second interface, wherein the third interface is the interface displayed when an operation is received on the first button.

8. The method according to any one of claims 1-7, characterized in that, When the second message and the third message belong to the first scenario, the fourth message is a reply message provided by the second voice assistant based on the second message and the fourth message. The second voice assistant is used to process the dialogue messages in the first scenario. The fourth message is surrounded by the identifier of the second voice assistant.

9. The method according to claim 8, characterized in that, Following the display of the third interface, the following is also included: After receiving a third operation on the first element in the second interface, a sixth message and a seventh message are displayed in the first window. The seventh message is surrounded by the identifier of the third voice assistant. The sixth message is an image generated based on the third operation. When the sixth message belongs to the message in the second scenario, the seventh message is a reply message provided by the third voice assistant based on the sixth message. The third voice assistant is used to process dialogue messages in the second scenario.

10. The method according to claim 8 or 9, characterized in that, The first window also includes: a second button, and after the third interface is displayed, it also includes: Upon receiving an operation on the second button, a fifth interface is displayed, which includes a third button related to the second voice assistant. Upon receiving an operation on the third button, a sixth interface is displayed, which includes a third window for processing dialogue messages in the first scenario. Upon receiving an operation to input the second message and the third message into the third window, the third window displays the second message, the third message, and the fourth message.

11. The method according to any one of claims 1-10, characterized in that, The first window also includes: a fourth button for inputting a second element in the first window, the second element including: an image, video, or document; and after the third interface is displayed, it further includes: Upon receiving an operation on the fourth button, one or more of the following are displayed in the first window: a button for viewing images or videos, a button for activating the camera, and a button for viewing documents.

12. The method according to any one of claims 1-11, characterized in that, After displaying the first interface, the following is also included: Upon receiving a fourth operation to drag the first element to the edge of the screen, a seventh interface is displayed, which includes: the content displayed in the first interface, and a fifth button for activating the first voice assistant. While the fourth operation continues to act on the first element, upon receiving an operation to drag the fourth element to the location of the fifth button, the first element is displayed in the first window.

13. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 12.

Images