Method for collaborative use of virtual assistant, and electronic device

Abstract

Provided in the present application are a method for collaborative use of a virtual assistant, and an electronic device. The method comprises: displaying a first page of a first application program; waking up a virtual assistant on the first page, so as to display a first window corresponding to the virtual assistant, and at least one first control adjacent to the first window, wherein the first control comprises a service recommendation matching the first page; in response to a first operation for the first page, updating the first page, so as to obtain a first updated page in the first application program; and updating the first control on the basis of the first updated page, so as to obtain a first updated control, wherein the first updated control comprises a service recommendation matching the first updated page. The present application can realize collaborative use of an application program and a virtual assistant, thereby improving the operation experience of users.

Description

Virtual assistant collaborative use method and electronic device TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to a virtual assistant collaborative use method and an electronic device. BACKGROUND

[0002] A virtual assistant is an interactive service realized through artificial intelligence technology. When a user uses an application of an electronic device, after waking up the virtual assistant, if the user operates the application again, the virtual assistant ends the conversation.

[0003] That is, at present, the virtual assistant only supports single wake-up and performs operations in its program, and the application and the virtual assistant do not support collaborative use, which affects the user experience.

[0004] SUMMARY

[0005] The present application provides a virtual assistant collaborative use method and an electronic device, which can realize collaborative use of an application and a virtual assistant and improve the user operation experience.

[0006] In a first aspect, the present application provides a virtual assistant collaborative use method applied to an electronic device, including: displaying a first page of a first application; waking up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control including service recommendations matched with the first page; in response to a first operation on the first page, updating the first page to obtain a first updated page in the first application; updating the first control based on the first updated page to obtain a first updated control, the first updated control including service recommendations matched with the first updated page.

[0007] The method shown in the present application supports operating a page in the first application again after waking up the virtual assistant in the first application, that is, the first application and the virtual assistant can be collaboratively operated, and the virtual assistant can provide service recommendations matched with the updated page, and the present application improves the user experience in using the virtual assistant.

[0008] In an implementation, the virtual assistant is woken up on the first page, including: in response to a second operation on a navigation button, the virtual assistant is woken up on the first page, wherein the navigation button includes at least one of a navigation bar, a home button in a navigation bar, and the second operation includes at least one of a long press operation and a click operation; or, in response to a third operation on a power key, the virtual assistant is woken up on the first page, and the third operation includes at least one of a long press operation and a click operation; or, in response to an operation of dragging target page content of the first page to the navigation button, the virtual assistant is woken up on the first page. With this implementation, multiple ways of waking up the virtual assistant are shown to meet different user habits and improve the user experience in using the virtual assistant.

[0009] In an implementation, in response to an operation of dragging target page content of the first page to the navigation button, the virtual assistant is woken up on the first page, including: in response to a selection operation on the target page content, a first selected area is generated at a location where the target page content is located; in response to a long press operation on the first selected area, a first moving area is generated, and the first moving area is used to copy and move the target page content; in response to an operation of dragging the first moving area to the navigation button, a transition window is displayed at a first location corresponding to the navigation button, and the transition window is used to prompt the user that the virtual assistant can be woken up at the first location; in a case where the first moving area reaches a first preset area, the virtual assistant is woken up on the first page, wherein the first preset area is an area adjacent to the transition window, and the first preset area is used to avoid a screen hot area. With this implementation, the virtual assistant can be woken up based on the dragging manner, and the screen hot area can be avoided in this process, thereby improving the user experience in using the virtual assistant.

[0010] In an implementation, in a case where the first moving area reaches the first preset area, the virtual assistant is woken up on the first page, and then, the method further includes: in response to an operation of dragging the first moving area to an adjacent area of the first window, a second control is generated in the adjacent area of the first window, and the second control is used to cache the target page content. With this implementation, the target page content dragged by the user can be cached for subsequent use by the user, and this setting manner meets the user's operation habits and improves the user experience in using the virtual assistant.

[0011] In an implementation, after the second control is generated in the adjacent area of the first window, the method further includes: updating the first control based on the target page content in the second control to obtain a second updated control, and the second updated control includes a service recommendation matched with the target page content. With this implementation, the service recommendation can be refreshed based on the target page content dragged by the user, so that the service recommendation meets the user's demand and improves the user experience in using the virtual assistant.

[0012] In an implementation manner, the first window corresponding to the virtual assistant is displayed, including: in the case that the virtual assistant is woken up in response to a second operation on the navigation button, or in the case that the virtual assistant is woken up in response to an operation of dragging the page content of the first page to the navigation button, the first window with a first input state is displayed, the first input state being a state that no voice input is currently received; in the case that the virtual assistant is woken up in response to a third operation on the power key, the first window with a second input state is displayed, the second input state being a state that voice input is currently received. With the implementation manner, the first window with different input states is provided based on different wake-up manners of the user, and the setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0013] In an implementation manner, the first window corresponding to the virtual assistant is displayed, including: in the case that the navigation bar is displayed on the first page, the first window is displayed in an adjacent area of the navigation bar to avoid the navigation bar. With the implementation manner, the first window and the navigation bar are avoided, and the setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0014] In an implementation manner, the first window corresponding to the virtual assistant is displayed, including: in the case that the navigation bar is displayed on the first page, the first window is displayed in an adjacent area of the navigation bar to avoid the navigation bar. With the implementation manner, the first window and the navigation bar are avoided, and the setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0015] In an implementation, the first window includes a first state window, a second state window, and a third state window; the first state window includes a first interaction area for receiving an input operation of the user; the second state window includes a first sub-window and a second sub-window, the first sub-window includes the first interaction area and a first panel displayed in a part of the display screen of the electronic device, the first panel is used for accommodating interaction content between the virtual assistant and the user, the second sub-window includes the first interaction area and a second panel displayed in a part of the display screen of the electronic device, and a panel height of the second panel is higher than a panel height of the first panel, the second panel is used for accommodating the interaction content between the virtual assistant and the user; the third state window includes the first interaction area and a third panel displayed in a full-screen area corresponding to at least one display screen, the third panel is used for accommodating the interaction content between the virtual assistant and the user. With the implementation, the first window in multiple states is provided, and the setting mode is suitable for the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0016] In an implementation, displaying the first window corresponding to the virtual assistant and at least one first control adjacent to the first window includes: displaying the first state window and displaying the first control in an outer peripheral adjacent area of the first state window; or, displaying the first sub-window and displaying the first control in the first panel, and the first control is displayed in an outer peripheral adjacent area of the first interaction area. With the implementation, the initial display mode of the first window is shown, and the setting mode is suitable for the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0017] In an implementation, after displaying the first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the method further includes: in response to an input operation on the first state window, displaying the first sub-window and displaying interaction content corresponding to the input operation in the first panel; in the first panel, if an area of a display area corresponding to the interaction content is greater than a first area threshold, displaying the second sub-window and displaying the interaction content in the second panel. With the implementation, the switching mode between the first window in different states is shown, and the setting mode is suitable for the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0018] In an implementation manner, after displaying the first window corresponding to the virtual assistant and the at least one first control adjacent to the first window, the method further includes: in response to a fourth operation on the first control, displaying a first sub-window and displaying interactive content corresponding to the first control in the first panel, the fourth operation including at least one of a click operation and a drag operation; and in the first panel, if an area of a display region corresponding to the interactive content is greater than a first area threshold, displaying a second sub-window and displaying the interactive content in a second panel. With this implementation manner, a switching manner between the first windows in different states is shown, and this setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0019] In an implementation manner, the method further includes: in a case where the first state window is displayed, in response to an upward sliding operation on a navigation bar in the first state window, displaying at least one of the first sub-window, the second sub-window, and the third state window; in a case where the first sub-window is displayed, in response to an upward sliding operation on a navigation bar in the first sub-window, displaying at least one of the second sub-window and the third state window; and in a case where the second sub-window is displayed, in response to an upward sliding operation on a navigation bar in the second sub-window, displaying the third state window. With this implementation manner, a switching manner between the first windows in different states is shown, and this setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0020] In an implementation manner, the method further includes: in a case where the third state window is displayed, in response to a downward sliding operation on a top region of the display screen, displaying at least one of the first state window, the first sub-window, and the second sub-window, or minimizing the third state window to a navigation button; in a case where the second sub-window is displayed, in response to a downward sliding operation on a navigation bar in the second sub-window, displaying at least one of the first state window and the first sub-window, or minimizing the second sub-window to the navigation button; and in a case where the first sub-window is displayed, in response to a downward sliding operation on a navigation bar in the first sub-window, displaying the first state window, or minimizing the first sub-window to the navigation button. With this implementation manner, a switching manner between the first windows in different states is shown, and this setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0021] In an implementation, after the first state window is displayed, the method further includes: in response to a long press and drag operation on the navigation bar in the first state window, displaying the first state window as a corresponding movement state window, the movement state window including the first interaction area and a fourth panel, the fourth panel being configured to accommodate the first control; and in response to a drag operation of the movement state window to a target position, displaying the movement state window at the target position. With this implementation, the manner of moving the first state window and the manner of changing the first state window during the movement are shown, which adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0022] In an implementation, after the second sub-window is displayed, the method further includes: in response to a long press and drag operation on the navigation bar in the second sub-window, reducing the panel height of the second panel to the panel height of the first panel, so that the second sub-window is transformed into the first sub-window; and in response to a drag operation of the first sub-window to a target position, displaying the first sub-window at the target position. With this implementation, the manner of moving the second sub-window and the manner of changing the second sub-window during the movement are shown, which adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0023] In an implementation, after the third state window is displayed, the method further includes: in response to a long press and drag operation on a top area of the display screen, displaying the third state window as the second state window; and in response to a drag operation of the second state window to a target position, displaying the second state window at the target position. With this implementation, the manner of moving the third state window and the manner of changing the third state window during the movement are shown, which adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0024] In an implementation, after the third state window is displayed, the method further includes: in a case where the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the third state window is displayed in a full-screen area corresponding to the entire display screen, in response to a long press and drag operation on a top area of the display screen along the top edge, displaying the third state window in a full-screen area corresponding to a part of the display screen based on a split-screen mode, and displaying the first page in a full-screen area corresponding to another part of the display screen. With this implementation, the manner of moving the third state window in the foldable electronic device and the manner of changing the third state window during the movement are shown, which adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0025] In an implementation, the method further includes: in response to the first operation on the first page, if the first status window is displayed on the first page, or if the second status window is displayed on the first page, or if the first status window and the second status window are not displayed on the first page, the display state of the first status window and the second status window remains unchanged. With this implementation, the display state of the first status window and the second status window remains unchanged during the operation on the first page, and thus the first application and the virtual assistant can operate cooperatively.

[0026] In an implementation, in response to the first operation on the first page, the first page is updated to obtain a first updated page in the first application, including: in response to the first operation on the first page, the first page is updated and the first control is hidden; and in response to the stopping of the first operation, the first updated page is obtained; wherein the first operation includes at least one of a swiping-up operation, a swiping-down operation, an operation of returning to a parent page, an operation of entering a child page, an operation of jumping to a second page and then returning to the first page, and the second page is a page other than the page in the first application. With this implementation, a hiding mechanism is provided during the updating of the first page, so as to avoid the impact of the non-adapted first control on the visual experience of the user. This setting mode adapts to the usage habits of the user and improves the usage experience of the user in using the virtual assistant.

[0027] In an implementation, the first control is updated based on the first updated page to obtain a first updated control, including: within a first preset time length after the stopping of the first operation, the first control is updated based on the first updated page; and in response to the expiration of the first preset time length, the first updated control is displayed. With this implementation, the service recommendation based on the first updated page can be matched within the preset time length, so as to ensure that the service recommendation in the first updated control is more adapted. This setting mode adapts to the usage habits of the user and improves the usage experience of the user in using the virtual assistant.

[0028] In an implementation, in response to the first operation on the first page, the first page is updated to obtain a first updated page in the first application, including: in response to the first operation on the first page, a first focus is set on the first page to focus on the first page, so as to control the first operation to be matched with the first page; and in response to the matching of the first operation with the first page, the first page is controlled to be updated corresponding to the first operation to obtain the first updated page. With this implementation, the user can operate in the desired area based on the setting of the focus. This setting mode adapts to the usage habits of the user and improves the usage experience of the user in using the virtual assistant.

[0029] In an implementation, in response to the first operation on the first page, the first focus is set on the first page to focus on the first page, and after the first window is displayed in the unfocused state, the first focus is moved to the first window to display the first window from the unfocused state to the focused state, the first window is focused in the case that the first window is displayed in the focused state, and the fifth operation is matched with the first window; in the case that the fifth operation is matched with the first window, the first window is controlled to respond to the fifth operation, and the fifth operation includes at least one of the up sliding operation, the down sliding operation, and the input operation. With the implementation, the user can be ensured to operate in the required area based on the moving focus, the setting mode adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0030] In an implementation, the first control is updated based on the first updated page to obtain the first updated control, including: in the case that the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the first updated page is displayed based on the split-screen mode, if the first focus stays in a first split-screen page corresponding to the first updated page, the first control is updated based on the first split-screen page to obtain the first updated control, and the first updated control includes service recommendations matched with the first split-screen page; if the first focus stays in a second split-screen page corresponding to the first updated page, the first control is updated based on the second split-screen page to obtain the first updated control, and the first updated control includes service recommendations matched with the second split-screen page. With the implementation, the user can be ensured to operate in the required area based on the moving focus, the setting mode adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0031] In an implementation, the first control is updated based on the first updated page to obtain the first updated control, including: in the case that the first updated page includes a floating window, and the first focus stays in the floating window, the first control is updated based on the floating window to obtain the first updated control, and the first updated control includes service recommendations matched with the floating window. With the implementation, the user can be ensured to operate in the required area based on the moving focus, the setting mode adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0032] In an implementation, after the first control is updated based on the first updated page to obtain the first updated control, the first window is minimized to the navigation button, and the interaction content is cached as historical content if the second preset time length is reached after the virtual assistant is woken up. With the implementation, the session of the virtual assistant can be automatically ended, the setting mode adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0033] In an implementation manner, after the first window is minimized to the navigation button and the interaction content is cached as the history content, the method further includes: waking up the virtual assistant again on the first page to display the first window corresponding to the virtual assistant, and displaying the history content in the first window. With the implementation manner, the history content can be retained in the virtual assistant, and the use demand of the user is met, and the operation habit of the user is met.

[0034] In an implementation manner, after the first control is updated based on the first updated page to obtain the first updated control, the method further includes: in response to the operation of swiping up from the bottom of the display screen of the electronic device, returning to the home screen page by the first application program, and ending the session of the virtual assistant. With the implementation manner, the home screen page can be returned based on the operation of the user, and the setting manner adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0035] In a second aspect, an embodiment of the present application provides a virtual assistant cooperative use device, applied to an electronic device, and including: a display module, the display module is configured to display a first page of a first application program; a wake-up module, the wake-up module is configured to wake up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control including a service recommendation matched with the first page; a page updating module, the page updating module is configured to update the first page in response to a first operation on the first page to obtain a first updated page in the first application program; and a control updating module, the control updating module is configured to update the first control based on the first updated page to obtain a first updated control, the first updated control including a service recommendation matched with the first updated page.

[0036] The device shown in the embodiment of the present application supports re-operation on the page in the first application program after the virtual assistant is woken up in the first application program, that is, the first application program and the virtual assistant can be cooperatively operated, and the virtual assistant can provide a service recommendation matched with the updated page, and the use experience of the user in the process of using the virtual assistant is improved.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory stores program instructions, when the program instructions are executed by the processor, the electronic device executes the virtual assistant cooperative use method provided in the first aspect and any possible implementation manner thereof.

[0038] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium includes computer instructions, when the computer instructions run on the electronic device, the electronic device executes the virtual assistant cooperative use method provided in the first aspect and any possible implementation manner thereof.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on an electronic device, causes the electronic device to perform the virtual assistant cooperative use method according to the first aspect and any possible implementation manner thereof.

[0040] It can be understood that the beneficial effects achieved by the technical solutions provided in the third aspect to the fifth aspect can refer to the beneficial effects in the first aspect and any possible implementation manner thereof, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0041] FIG. 1 is a schematic diagram of a scenario in which an electronic device wakes up a virtual assistant;

[0042] FIG. 2 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application;

[0043] FIG. 3 is a schematic diagram of a software structure of an electronic device according to an embodiment of the present application;

[0044] FIG. 4 is a schematic diagram of a first scenario in which an electronic device provides a guide flow according to an embodiment of the present application;

[0045] FIG. 5 is a schematic diagram of a second scenario in which an electronic device provides a guide flow according to an embodiment of the present application;

[0046] FIG. 6 is a schematic diagram of a third scenario in which an electronic device provides a guide flow according to an embodiment of the present application;

[0047] FIG. 7 is a schematic diagram of a fourth scenario in which an electronic device provides a guide flow according to an embodiment of the present application;

[0048] FIG. 8 is a schematic diagram of a fifth scenario in which an electronic device provides a guide flow according to an embodiment of the present application;

[0049] FIG. 9 is a schematic diagram of a sixth scenario in which an electronic device provides a guide flow according to an embodiment of the present application;

[0050] FIG. 10 is a schematic diagram of a scenario in which a virtual assistant is woken up for the first time according to an embodiment of the present application;

[0051] FIG. 11 is a schematic diagram of a scenario in which a window state of a virtual assistant is switched for the first time according to an embodiment of the present application;

[0052] FIG. 12 is a schematic diagram of a first scenario in which a virtual assistant is set according to an embodiment of the present application;

[0053] FIG. 13 is a schematic diagram of a second scenario in which a virtual assistant is set according to an embodiment of the present application;

[0054] FIG. 14 is a schematic diagram of a first wake-up mode of a virtual assistant in an application according to an embodiment of the present application;

[0055] FIG. 15 is a schematic diagram of a dynamic effect change mode of a navigation bar according to an embodiment of the present application;

[0056] FIG. 16 is a schematic diagram of a second wake-up mode of a virtual assistant in an application according to an embodiment of the present application;

[0057] FIG. 17 is a schematic diagram of a third wake-up mode of a virtual assistant in an application according to an embodiment of the present application;

[0058] FIG. 18 is a schematic diagram of a fourth wake-up mode of a virtual assistant in an application according to an embodiment of the present application;

[0059] FIG. 19 is a schematic diagram of a scenario of recognizing a user's intention by an electronic device according to an embodiment of the present application;

[0060] FIG. 20 is a schematic diagram of a fifth wake-up mode of a virtual assistant in an application according to an embodiment of the present application;

[0061] FIG. 21 is a schematic diagram of a display state of a first window according to an embodiment of the present application;

[0062] FIG. 22 is a schematic diagram of an input state of a first window according to an embodiment of the present application;

[0063] FIG. 23 is a schematic diagram of a display mode of a first control according to an embodiment of the present application;

[0064] FIG. 24 is a schematic diagram of a display mode of a second control according to an embodiment of the present application;

[0065] FIG. 25 is a schematic diagram of an update process of a first control according to an embodiment of the present application;

[0066] FIG. 26 is a schematic diagram of a scenario of input operation of a first state window by a user according to an embodiment of the present application;

[0067] FIG. 27 is a schematic diagram of a first scenario of window state switching of a first window according to an embodiment of the present application;

[0068] FIG. 28 is a schematic diagram of a second scenario of window state switching of a first window according to an embodiment of the present application;

[0069] FIG. 29 is a schematic diagram of a third scenario of window state switching of a first window according to an embodiment of the present application;

[0070] FIG. 30 is a schematic diagram of a fourth scenario of window state switching of a first window according to an embodiment of the present application;

[0071] FIG. 31 is a schematic view of a fifth scenario of window state switching of a first window according to an embodiment of the present application;

[0072] FIG. 32 is a schematic view of a sixth scenario of window state switching of a first window according to an embodiment of the present application;

[0073] FIG. 33 is a schematic view of a seventh scenario of window state switching of a first window according to an embodiment of the present application

[0074] FIG. 34 is a schematic view of a first scenario of moving a first window according to an embodiment of the present application;

[0075] FIG. 35 is a schematic view of a second scenario of moving a first window according to an embodiment of the present application;

[0076] FIG. 36 is a schematic view of a third scenario of moving a first window according to an embodiment of the present application;

[0077] FIG. 37 is a schematic view of a fourth scenario of moving a first window according to an embodiment of the present application;

[0078] FIG. 38 is a schematic view of a fifth scenario of moving a first window according to an embodiment of the present application;

[0079] FIG. 39 is a schematic view of a first scenario of user interaction with a virtual assistant based on a first control according to an embodiment of the present application;

[0080] FIG. 40 is a schematic view of a second scenario of user interaction with a virtual assistant based on a first control according to an embodiment of the present application;

[0081] FIG. 41 is a schematic view of a third scenario of user interaction with a virtual assistant based on a first control according to an embodiment of the present application;

[0082] FIG. 42 is a schematic view of a first scenario of user operation on a first page according to an embodiment of the present application;

[0083] FIG. 43 is a schematic view of a second scenario of user operation on a first page according to an embodiment of the present application;

[0084] FIG. 44 is a schematic view of a third scenario of user operation on a first page according to an embodiment of the present application;

[0085] FIG. 45 is a schematic view of a fourth scenario of user operation on a first page according to an embodiment of the present application;

[0086] FIG. 46 is a schematic view of a first scenario of focus switching according to an embodiment of the present application;

[0087] FIG. 47 is a schematic view of a second scenario of focus switching according to an embodiment of the present application;

[0088] FIG. 48 is a schematic diagram of a third scenario of focus switching according to an embodiment of the present application;

[0089] FIG. 49 is a schematic diagram of a fourth scenario of focus switching according to an embodiment of the present application;

[0090] FIG. 50 is a schematic diagram of a first scenario of ending a virtual assistant session according to an embodiment of the present application;

[0091] FIG. 51 is a schematic diagram of a second scenario of ending a virtual assistant session according to an embodiment of the present application;

[0092] FIG. 52 is a schematic diagram of a scenario of reawakening a virtual assistant according to an embodiment of the present application;

[0093] FIG. 53 is a schematic diagram of a structure of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0094] The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0095] In the description of the present application, unless otherwise specified, “ / ” means “or”, for example, A / B can mean A or B. “And / or” in the present application is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, “at least one” means one or more, and “multiple” means two or more. “First”, “second”, and the like do not limit the quantity and execution order, and “first”, “second”, and the like do not necessarily mean different.

[0096] It should be noted that in the present application, “exemplary” or “for example” is used to mean example, illustration, or description. Any embodiment or design scheme described as “exemplary” or “for example” in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, “exemplary” or “for example” is used to present the relevant concept in a specific manner.

[0097] Virtual assistants are usually provided in electronic devices, which are interactive services realized through artificial intelligence technology and can simulate the role of human assistants to quickly handle daily affairs. Virtual assistants include voice assistants and large model assistants, etc. Voice assistants can understand and respond to user voice instructions using voice recognition technology, providing functions such as information query, schedule management, and device control. Large model assistants can handle complex tasks and data, providing functions such as text generation, image recognition, and data analysis. These types of assistants can meet different interactive needs of users. The virtual assistant involved in the present application can be any type of virtual assistant, which is not limited in the present application.

[0098] FIG. 1 is a schematic diagram of a scenario in which an electronic device wakes up a virtual assistant.

[0099] As shown in A of FIG. 1, an example is illustrated in a chat scenario. The user can press the power key 3 during the chat in the chat interface 2 of the chat application in the electronic device 1. As shown in B and C of FIG. 1, in response to the user's pressing operation on the power key 3, the electronic device 1 can wake up the virtual assistant at the bottom of the chat interface 2 to display a first animation 4 corresponding to the virtual assistant. The user can input a voice instruction based on the first animation 4 to make the virtual assistant execute the operation in its program based on the voice instruction, for example, an operation of querying a route. As shown in D of FIG. 1, if the user operates the chat interface 2 again, the virtual assistant ends the session and returns to the chat interface 2 as shown in A of FIG. 1.

[0100] Correspondingly, in other types of application programs, if the virtual assistant is woken up and the application program is operated again, the virtual assistant ends the session.

[0101] That is, at present, the virtual assistant only supports single wake-up and executes operations in its program, and the application program and the virtual assistant do not support collaborative use, which affects the user experience.

[0102] To solve the above problems, the present application embodiment provides a virtual assistant collaborative use method.

[0103] The virtual assistant collaborative use method provided by the present application embodiment can be applied to an electronic device. The electronic device includes but is not limited to a mobile phone, a tablet computer, a personal computer, a workstation device, a large-screen device (such as a smart screen, a smart television, etc.), a wearable device (such as a smart bracelet, a smart watch), a palm game console, a home game console, a virtual reality device, an augmented reality device, a mixed reality device, a vehicle-mounted intelligent terminal, etc. The present application embodiment does not limit the specific technology and specific device form of the electronic device.

[0104] FIG. 2 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.

[0105] As shown in FIG. 2, the electronic device 900 can include a processor 910, a memory 920, a Universal Serial Bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 01, an antenna 02, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, a headphone jack 970D, a sensor module 980, a key 990, a motor 991, a camera 992, a display screen 993, and a Subscriber Identification module (SIM) card interface 994, etc. The sensor module 980 can include a touch sensor 980A, a gyroscope sensor 980B, a barometric sensor 980C, a geomagnetic sensor 980D, an acceleration sensor 980E, a distance sensor 980F, a proximity light sensor 980G, a fingerprint sensor 980H, a temperature sensor 980J, etc. The gyroscope sensor 980B, the barometric sensor 980C, the geomagnetic sensor 980D, the acceleration sensor 980E, etc. can be used to detect the motion state of the electronic device, and thus can also be referred to as motion sensors.

[0106] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 900. In other embodiments of the present application, the electronic device 900 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0107] The processor 910 can include one or more processing units, for example: the processor 910 can include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a Neural-network Processing Unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0108] The memory 920 can be used to store computer-executable program code, including instructions. The memory 920 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs (such as a sound playing program, an image playing program, etc.) required by at least one function, and the like. The data storage area can store data (such as audio data, a phone book, etc.) created during use of the electronic device 900, and the like. In addition, the memory 920 can include a high-speed random access memory, and can further include a nonvolatile memory such as at least one of a magnetic disk storage device, a flash memory device, a Universal Flash Storage (UFS), and the like. The processor 910 executes various function applications and data processing of the electronic device 900 by running instructions stored in the memory 920 and / or instructions stored in a memory disposed in the processor.

[0109] The USB interface 930 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 930 can be used to connect a charger to charge the electronic device 900, and can be used to transmit data between the electronic device 900 and a peripheral device. The interface can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as an AR device, and the like.

[0110] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 900. In some other embodiments of the present application, the electronic device 900 can also use different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0111] The charging management module 940 is configured to receive a charging input from a charger.

[0112] The power management module 941 is configured to connect the battery 942 and the charging management module 940 to the processor 910.

[0113] The wireless communication function of the electronic device 900 can be implemented through the antenna 01, the antenna 02, the mobile communication module 950, the wireless communication module 960, a modem processor, and a baseband processor, and the like.

[0114] The mobile communication module 950 can provide a solution including 2G / 3G / 4G / 5G wireless communication applied to the electronic device 900.

[0115] The wireless communication module 960 can provide a solution for wireless communication, including Wireless Local Area Networks (WLAN) (e.g., Wireless Fidelity (Wi-Fi) network), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR) technology, etc., which are applied to the electronic device 900.

[0116] The wireless communication module 960 can be one or more devices that integrate at least one communication processing module. The wireless communication module 960 receives an electromagnetic wave via the antenna 02, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 910. The wireless communication module 960 can also receive a signal to be transmitted from the processor 910, frequency-modulate it, amplify it, and convert it into electromagnetic wave radiation via the antenna 02.

[0117] In some embodiments, the antenna 01 of the electronic device 900 is coupled with the mobile communication module 950, and the antenna 02 is coupled with the wireless communication module 960, so that the electronic device 900 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), Beidou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0118] In the embodiments of the present application, the electronic device 900 needs to realize the normal interaction of the virtual assistant under the condition that the mobile network switch is turned on and the WLAN is turned on.

[0119] The electronic device 900 realizes the display function through the GPU, the display screen 993, and the application processor, etc. The GPU is a microprocessor for image processing, which is connected with the display screen 993 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 910 can include one or more GPUs, which execute program instructions to generate or change display information.

[0120] In the embodiments of the present application, the electronic device 900 can process the first window, the first control, the first page, etc. through the GPU, and send the display. The display screen 993 can display the first window, the first control, and the first page.

[0121] The display screen 993 is configured to display images, videos, and the like. The display screen 993 includes a display panel. The display panel can be a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic Light Emitting Diode (AMOLED), a Flex Light-Emitting Diode (FLED), a Miniled, a Micro Led, a Micro-oLed, a Quantum dot Light Emitting Diodes (QLED), or the like. In some embodiments, the electronic device 900 can include one or N display screens 993, where N is a positive integer greater than 1.

[0122] In some embodiments, the electronic device 900 can be a foldable electronic device. When the electronic device 900 is a foldable electronic device, the electronic device 900 can include at least two display screens 993.

[0123] The electronic device 900 can implement a photographing function through an ISP, a camera 992, a video codec, a GPU, a display screen 993, and an application processor.

[0124] The ISP is configured to process data fed back by the camera 992. For example, when taking a photo, the shutter is opened, and light is transmitted to the camera photosensitive element through the lens, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 992.

[0125] The camera 992 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, RYYB, YUV, or the like format. In some embodiments, the electronic device 900 can include one or N cameras 992, where N is a positive integer greater than 1.

[0126] The electronic device 900 can implement audio functions through an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, a headset jack 970D, and an application processor, etc. For example, music playing, recording, etc.

[0127] In embodiments of the present application, the electronic device 900 can implement the wake-up of the virtual assistant through the audio module 970.

[0128] The touch sensor 980A, also referred to as a "touch device". The touch sensor 980A can be disposed on the display screen 993, and the touch sensor 980A and the display screen 993 form a touch screen, also referred to as a "touch screen". The touch sensor 980A is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. The display screen 993 can provide visual output related to the touch operation. In other embodiments, the touch sensor 980A can also be disposed on the surface of the electronic device 900, which is different from the position of the display screen 993.

[0129] In embodiments of the present application, the touch sensor 980A can detect the user's click operation, long press operation, drag operation, etc.

[0130] The gyroscope sensor 980B can be used to determine the motion posture of the electronic device 900.

[0131] The barometric sensor 980C is used to measure air pressure. In some embodiments, the electronic device 900 calculates the altitude, assists in positioning and navigation by measuring the air pressure value through the barometric sensor 980C.

[0132] The geomagnetic sensor 980D includes a Hall sensor. The electronic device 900 can use the geomagnetic sensor 980D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 900 is a flip phone, the electronic device 900 can detect the opening and closing of the flip cover according to the geomagnetic sensor 980D. Further, according to the detected opening and closing state of the cover or the opening and closing state of the flip cover, the electronic device 900 can set the flip cover automatic unlocking and other features.

[0133] The acceleration sensor 980E can detect the magnitude of the acceleration of the electronic device 900 in each direction (generally three axes). When the electronic device 900 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the electronic device posture, applied to landscape / portrait screen switching, pedometer, etc.

[0134] The distance sensor 980F is used to measure distance. The electronic device 900 can measure distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 900 can use the distance sensor 980F to measure distance to achieve fast focusing.

[0135] The proximity light sensor 980G can include, for example, a light emitting diode and a light detector, such as a photodiode.

[0136] The fingerprint sensor 980H is configured to collect a fingerprint. The electronic device 900 can implement fingerprint unlocking, access application lock, fingerprint photograph, fingerprint answer a call, and the like, by using the collected fingerprint characteristics.

[0137] The temperature sensor 980J is configured to detect a temperature.

[0138] The keys 990 include a power key, a volume key, and the like. The keys 990 can be mechanical keys. Alternatively, the keys 990 can be touch keys. The electronic device 900 can receive a key input and generate a key signal input related to user settings and function control of the electronic device 900.

[0139] In an embodiment of the present application, the keys 990 can be used to wake up the virtual assistant.

[0140] The motor 991 can generate a vibration prompt. The motor 991 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. For example, touch operations on different applications (such as photographing, audio playing, and the like) can correspond to different vibration feedback effects. The motor 991 can also correspond to different vibration feedback effects for touch operations on different regions of the display screen 993. Different application scenarios (such as time reminders, receiving information, alarms, games, and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customizable.

[0141] In an embodiment of the present application, the motor 991 can provide vibration feedback to the user when the user performs an interactive operation, thereby improving the user's use experience.

[0142] The SIM card interface 994 is configured to connect a SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 994 to realize contact and separation with the electronic device 900.

[0143] The software system of the electronic device 900 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. In an embodiment of the present application, an Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device 900.

[0144] FIG. 3 is a schematic diagram of a software structure of an electronic device according to an embodiment of the present application.

[0145] A layered architecture divides software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0146] The application layer can include a series of application packages.

[0147] As shown in FIG. 3, the application package can include battery management, camera, gallery, calendar, call, map, navigation, music, video, short message, etc.

[0148] In an embodiment of the present application, the first application can be any type of application described above.

[0149] The application framework layer provides application programs in the application layer with application program interfaces (APIs) and programming frameworks. The application framework layer includes some pre-defined functions.

[0150] As shown in FIG. 3, the application framework layer can include a window manager, an input manager InputManager, a sensor manager SensorManager, a phone manager, a resource manager, a notification manager, etc.

[0151] The input manager can be used to listen to user input events, such as user finger click events, sliding events, etc. on the display screen 993 of the electronic device 900. By listening to the input events, the electronic device 900 can determine whether the electronic device is being used.

[0152] The sensor manager is used to listen to data returned by various sensors in the electronic device, such as motion sensor data, proximity light sensor data, temperature sensor data, etc. Using the data returned by various sensors, the electronic device can determine whether it is shaking, or whether the display screen 993 is blocked, etc.

[0153] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0154] The core library contains two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0155] The application program layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application program layer and the application framework layer as binary files. The virtual machine is used to perform functions such as management of object life cycle, stack management, thread management, management of security and exceptions, and garbage collection.

[0156] The system library can include a plurality of functional modules. For example, a surface manager (Surface Manager), media libraries, a three-dimensional graphics processing library (for example, OpenGL ES), a 2D graphics engine (for example, SGL), and the like.

[0157] The surface manager is used to manage a display subsystem and provides fusion of 2D and 3D layers for a plurality of application programs.

[0158] The media library supports playback and recording of a plurality of commonly used audio, video formats, and static image files. The media library can support a plurality of audio and video encoding formats, for example, MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like.

[0159] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, and the like.

[0160] The 2D graphics engine is a drawing engine for 2D drawing.

[0161] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, a power supply driver, an audio driver, and a sensor driver.

[0162] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 900. In other embodiments of the present application, the electronic device 900 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0163] In order to realize the collaborative use of the virtual assistant, after the electronic device 900 is powered on, a guide flow can be provided to the user to enable the user to learn the operation mode of the virtual assistant and complete the initial setting of the virtual assistant.

[0164] It should be noted that the guide flow can be provided when the electronic device 900 is first enabled or when the electronic device 900 is deactivated for a period of time and then triggers the virtual assistant again. The specific manner in which the electronic device 900 provides the guide flow is not limited in the embodiments of the present application.

[0165] FIG. 4 is a first scenario diagram illustrating that the electronic device provides a guide flow according to an embodiment of the present application.

[0166] As shown in FIG. 4, an example is taken to illustrate a case where the electronic device 900 is started for the first time. After the electronic device 900 is powered on, a first boot page 10 is displayed. The first boot page 10 can include a virtual assistant enabling control 11 for enabling the virtual assistant. The virtual assistant enabling control 11 can include a first option control 12 therein.

[0167] The first option control 12 can be in a checked mode by default, so that the electronic device 900 can enable the virtual assistant. If the first option control 12 is not in the checked mode, the user can click the first option control 12 to check it, so that the electronic device 900 can enable the virtual assistant.

[0168] The first boot page 10 can further include a first description area 13 for describing matters needing attention for enabling the virtual assistant.

[0169] In an implementation manner, the first description area 13 can include an out of box experience (OOBE) description. The content of the OOBE description can include, for example, “The virtual assistant and its underlying service (fast application engine) need to be connected to the network, and need to enable local recording, reading audio, video, making phone calls, sending short messages, reading short messages, reading setting unique identifiable information, reading positioning information, reading phonebook data, reading user local number, reading calendar, reading application list, reading call record, opening mobile network switch, enabling WLAN, modifying calendar, modifying short message, enabling system notification, collecting your biometric information, this device only collects personal information related to the above behaviors when specific business functions are used, this service provides users with intelligent voice interaction basic business functions, and also provides more extended business functions, closing the extended business does not affect the use of the basic business functions, enabling the service means that the user agrees to the virtual assistant user agreement permission usage description, and knows the statement about the virtual assistant and privacy.” The specific content of the OOBE description is not limited in the embodiments of the present application.

[0170] The first boot page 10 further includes a next step button 14. The user can click the next step button 14, and in response to the clicking operation of the user on the next step button 14, the electronic device 900 enters a second boot page 15.

[0171] The second boot page 15 can include a second description area 16 for describing the characteristics of the virtual assistant.

[0172] For example, the characteristic description content in the second description area 16 can include "long press the navigation prompt bar or the home button to call the virtual assistant at any time, and any text, picture or file on the screen can be dragged to the virtual assistant to complete the corresponding question and answer or service provision, etc.". The embodiment of the present application does not limit the characteristic description content of the virtual assistant.

[0173] The second guide page 15 further includes a confirmation button 17. The user can click the confirmation button 17, and in response to the clicking operation of the user on the confirmation button 17, the electronic device 900 enters a third guide page.

[0174] In the above scenario, the electronic device 900 exemplarily describes the matters needing attention and characteristics of the virtual assistant in the guide flow.

[0175] Further, the electronic device 900 provides operation teaching to the user in the guide flow.

[0176] FIG. 5 is a second scenario diagram of the electronic device providing a guide flow according to an embodiment of the present application.

[0177] As shown in FIG. 5, the electronic device 900 can enter a third guide page 18, and the third guide page 18 can include a third description area 19 for describing the function of the virtual assistant.

[0178] For example, the function description content of the third description area 19 can include "wake up the intelligent voice, search the encyclopedia immediately, make a phone call or set an alarm, etc.". The embodiment of the present application does not limit the function description content of the virtual assistant.

[0179] The third guide page 18 further includes a confirmation button 20. The user can click the confirmation button 20, and in response to the clicking operation of the user on the confirmation button 20, the electronic device 900 enters a fourth guide page 21.

[0180] The fourth guide page 21 can include a first teaching area 22. The first teaching area 22 can be used to enable the user to learn the way of waking up the virtual assistant based on the breath.

[0181] The first teaching area 22 can include a first image example area 23 and a first text example area 24. The first image example area 23 can enable the user to learn the way of waking up the virtual assistant based on the breath through images. The first text example area 24 can enable the user to learn the way of waking up the virtual assistant based on the breath through text.

[0182] For example, the first image example area 23 can include an image for representing that the user performs voice input on the bottom microphone of the electronic device 900.

[0183] For example, the text teaching content of the first text example area 24 can include "Put the electronic device close to the mouth (within 3 cm) and directly speak the voice instruction to the microphone."

[0184] The fourth guide page 21 further includes a next step button 25. The user can click the next step button 25, and in response to the clicking operation of the user on the next step button 25, the electronic device 900 enters a fifth guide page 26.

[0185] The fifth guide page 26 can include a second teaching area 27, which can enable the user to learn the way of waking up the virtual assistant based on the power key.

[0186] The second teaching area 27 can include a second image example area 28 and a second text example area 29. The second image example area 28 can enable the user to learn the way of waking up the virtual assistant based on the power key through images. The second text example area 29 can enable the user to learn the way of waking up the virtual assistant based on the power key through texts.

[0187] For example, the second image example area 28 can include an image for representing that the user wakes up the virtual assistant by long pressing the power key for 1 second.

[0188] For example, the text teaching content of the second text example area 29 can include "Long press the power key for 1 second to wake up the virtual assistant, and speak after releasing the power key; long press the power key for 3 seconds to shut down the electronic device."

[0189] The fifth guide page 26 further includes a next step button 30. The user can click the next step button 30, and in response to the clicking operation of the user on the next step button 30, the electronic device 900 enters a sixth guide page 31.

[0190] The sixth guide page 31 can include a first input area 32 and a first input animation 33. The first input area 32 can guide the user to input a voice instruction to enable the virtual assistant to record the voiceprint of the user.

[0191] For example, the guiding input content of the first input area 32 can include "Keep the electronic device 30 cm away from the face, keep the environment quiet, and clearly speak the instruction."

[0192] The user can perform a corresponding voice input operation according to the guiding input content, and in the process of the voice input operation of the user, the electronic device 900 can display the first input animation 33 as a jumping animation effect to represent that the voice input is currently being received. It should be noted that the first input animation 33 can also be displayed as a flashing animation effect, and the type of the animation effect of the first input animation 33 is not limited in the embodiments of the present application.

[0193] After the electronic device 900 completes recording the voiceprint of the user, the electronic device 900 can jump to the first trial page.

[0194] In the above scenario, the electronic device 900 exemplarily illustrates the wake-up manner involving the virtual assistant in the boot flow.

[0195] Further, the electronic device 900 provides the first trial teaching to the user in the boot flow.

[0196] FIG. 6 is a third scenario diagram of the electronic device providing a boot flow according to an embodiment of the present application.

[0197] As shown in FIG. 6, the electronic device 900 can enter the first trial page 34, which can include a trial button 35.

[0198] The user can click the trial button 35, and in response to the click operation of the user on the trial button 35, the electronic device 900 enters the second trial page 36.

[0199] The electronic device 900 can display a navigation prompt bar 37 at the bottom of the second trial page 36, and display a first prompt box 38 at the top of the second trial page 36, the first prompt box 38 being used to prompt the user of the wake-up manner of the virtual assistant.

[0200] For example, the prompt content of the first prompt box 38 can include “long press the bottom navigation bar to wake up the virtual assistant”.

[0201] The user can long press the navigation prompt bar 37, and in response to the long press operation of the user on the navigation prompt bar 37, the electronic device 900 hides the navigation prompt bar 37 at the bottom of the second trial page 36, and displays a first trial window 39 corresponding to the virtual assistant and at least one first trial control 40 adjacent to the first trial window 39. The first trial control 40 is used to provide a service recommendation matched with the second trial page 36. The service recommendation can be, for example, a full-text translation, a summary, and the like. The service recommendation is specifically described in subsequent embodiments of the present application, which will not be described herein.

[0202] It should be noted here that the first trial window 39 is displayed in a default state when it is first woken up. The type of window state of the virtual assistant is specifically described in subsequent embodiments of the present application, which will not be described herein.

[0203] The electronic device 900 can also update the first prompt box 38 at the top of the second trial page 36 to generate a second prompt box 41, the second prompt box 41 being used to prompt the user that the virtual assistant has been woken up.

[0204] For example, the prompt content of the second prompt box 41 can include "shortcut to wake up the virtual assistant for content understanding."

[0205] After the second prompt box 41 is displayed for a certain time length, the electronic device 900 can update the second prompt box 41 on the second trial page 36, generate a third prompt box 42, and the third prompt box 42 is used to prompt the user to exit the virtual assistant. For example, the display time length of the second prompt box 41 can be 2 seconds.

[0206] It should be noted that the way in which the electronic device 900 updates the second prompt box 41 on the second trial page 36 to generate the third prompt box 42 can be based on a transition effect. The specific updating manner of the page content is not limited in the embodiments of the present application.

[0207] For example, the prompt content of the third prompt box 42 can include "slide down from the top window edge of the first trial window to manually exit the virtual assistant."

[0208] The user can perform a slide-down operation from the top window edge of the first trial window 39 to end the conversation of the virtual assistant. Correspondingly, the electronic device 900 updates the third prompt box 42 at the top of the second trial page 36 to generate a fourth prompt box 43, and the fourth prompt box 43 is used to prompt the user that the virtual assistant has been manually exited.

[0209] For example, the prompt content of the fourth prompt box 43 can include "manually exited the virtual assistant."

[0210] In the above scenario, the electronic device 900 provides the user with trial teaching of gesture navigation in the guide flow. It should be noted that the embodiments of the present application do not provide trial teaching of all operation modes, and the trial teaching of other modes can be adapted based on the specific operation mode, for example, the trial teaching of the navigation bar can be individually adapted based on the specific operation mode, and the specific mode of the trial teaching is not limited in the embodiments of the present application.

[0211] Further, the electronic device 900 provides the user with a second trial teaching in the guide flow.

[0212] FIG. 7 is a fourth scenario diagram of the electronic device providing a guide flow according to an embodiment of the present application.

[0213] As shown in FIG. 7, the electronic device 900 can enter a third trial page 44, display a fifth prompt box 45 at the top of the third trial page 44, and display a navigation prompt bar 37 at the bottom of the third trial page 44, wherein the fifth prompt box 45 is used to prompt the user to wake up the virtual assistant.

[0214] For example, the third trial page 44 can be a page in a document application, and a plurality of documents can be provided in the page. For example, the first document 46 is included. The prompt content of the fifth prompt box 45 can include "Please long press and drag the first document to the bottom navigation bar."

[0215] The user can long press the first document 46 and drag the first document 46 to the navigation prompt bar 37. In response to the user dragging the first document 46 to the navigation prompt bar 37, the electronic device 900 displays the first trial window 39 corresponding to the virtual assistant at the bottom of the third trial page 44, and at least one first trial control 40 adjacent to the first trial window 39.

[0216] It should be noted that the first trial window 39 has different state types. In the current scenario, the first trial window 39 is in a state that can accommodate interactive content, which is different from the default state displayed in the foregoing embodiments. The state types of the window corresponding to the virtual assistant are described in subsequent embodiments of the present application, and the embodiments of the present application will not be described here.

[0217] In addition, the electronic device can also update the fifth prompt box 45 at the top of the third trial page 44 to generate the sixth prompt box 47, and the sixth prompt box 47 is used to prompt the user that the virtual assistant has been awakened.

[0218] For example, the prompt content of the sixth prompt box 47 can include "The virtual assistant has been quickly awakened for content understanding."

[0219] In the above scenario, the electronic device 900 provides the user with a trial teaching in the guide flow that the page content is dragged to the navigation prompt bar 37 to awaken the virtual assistant without awakening the virtual assistant.

[0220] Further, the electronic device 900 provides the user with a third trial teaching in the guide flow.

[0221] FIG. 8 is a fifth scenario diagram of the electronic device providing a guide flow according to an embodiment of the present application.

[0222] As shown in FIGS. 7 and 8, after the sixth prompt box 47 is displayed for a certain time length, the electronic device 900 can restore the first trial window 39 from the state that can accommodate interactive content to the default state on the third trial page 44, and update the sixth prompt box 47 at the top of the third trial page 44 to generate the seventh prompt box 48. The seventh prompt box 48 is used to prompt the user to long press and drag the first document 46 to the virtual assistant. For example, the display time length of the sixth prompt box 47 can be 2 seconds.

[0223] It should be noted that the electronic device 900 restores the first trial window 39 from the state capable of containing interactive content to the default state on the third trial page 44, and updates the sixth prompt box 47 at the top of the third trial page 44 to generate the seventh prompt box 48, which can be implemented based on a transition effect. The specific updating manner of the page content in the embodiments of the present application is not limited.

[0224] For example, the prompt content of the seventh prompt box 48 can include “Please long press and drag the first document to the virtual assistant.”

[0225] The user can long press the first document 46 and perform a drag operation on the first document 46 to the first trial window 39. In response to the operation of the user dragging the first document 46 to the first trial window 39, the electronic device 900 displays the first trial window 39 from the default state to the state capable of containing interactive content, and displays at least one first trial control 40 adjacent to the first trial window 39.

[0226] In addition, the electronic device can also update the seventh prompt box 48 at the top of the third trial page 44 to generate an eighth prompt box 49, and the eighth prompt box 49 is used to prompt the user that the current trial teaching has been completed.

[0227] For example, the prompt content of the eighth prompt box 49 can include “All operations have been completed.”

[0228] In the above scenario, the electronic device 900 provides the user with trial teaching of dragging page content to the virtual assistant in the wake-up state of the virtual assistant and the virtual assistant providing corresponding services in the guide flow.

[0229] Further, the electronic device 900 provides a flow summary in the guide flow.

[0230] FIG. 9 is a sixth scenario diagram of an electronic device providing a guide flow according to an embodiment of the present application.

[0231] As shown in FIGS. 8 and 9, after the eighth prompt box 49 is displayed for a certain time length, the electronic device 900 can display a first summary page 50, and display a first summary area 51 in the first summary page 50, and the first summary area 51 is used to summarize the functions of the virtual assistant.

[0232] For example, the function summary content of the first summary area 51 can include “understand screen content, automate execution, and intelligent agent store.” The function summary content of the virtual assistant is not limited in the embodiments of the present application.

[0233] It should be noted that, in any page of the above guiding process, an exit button can be provided to enable the user to end the guiding process at any process, and in any page, an up button can be provided to enable the user to return to the previous page.

[0234] It should be noted that, in any page of the above guiding process, when the first trial window 39 is displayed, the electronic device 900 displays the same animation effect (hereinafter referred to as animation effect) as in the actual operation process on the first trial window 39. The display mode of the animation effect is not specifically described in the above embodiment, and will be described in detail in subsequent embodiments of the present application.

[0235] Any page in the guiding process provided by the embodiments of the present application can increase or delete page elements according to actual conditions, and the specific display content of the guiding page is not limited by the embodiments of the present application.

[0236] After the user completes the guiding process, the electronic device 900 can use the virtual assistant.

[0237] FIG. 10 is a schematic diagram of a scenario of first waking up a virtual assistant according to an embodiment of the present application.

[0238] As shown in FIG. 10, the user can wake up the virtual assistant in various ways, for example, the voice-based way of waking up the virtual assistant as shown in the guiding process, the power key-based way of waking up the virtual assistant, the navigation bar-based way of waking up the virtual assistant, and the drag-based way of waking up the virtual assistant, etc. In addition to the ways in the examples, the Bluetooth key-based way of waking up the virtual assistant and the custom gesture-based way of waking up the virtual assistant, etc. can also be used. The way of waking up the virtual assistant is not limited by the embodiments of the present application.

[0239] The power key 52-based way of waking up the virtual assistant is exemplarily described.

[0240] The user can long press the power key 52 to wake up the virtual assistant for the first time. In order to ensure that the user understands the use of the virtual assistant, in response to the user long pressing the power key 52 to wake up the virtual assistant for the first time, the electronic device 900 displays a first navigation page 53 and displays a first navigation area 54 in the first navigation page 53, and the first navigation area 54 is used to describe the characteristics of the virtual assistant.

[0241] For example, the characteristic description content in the first navigation area 54 can include “long press the navigation prompt bar or the home button to wake up the virtual assistant at any time, and any text, picture, or file on the screen can be dragged to the virtual assistant to complete corresponding question and answer, service provision, etc.” The characteristic description content of the virtual assistant is not limited by the embodiments of the present application.

[0242] The first navigation page 53 further includes a trial button 55 and a quit button 56. The user can click the trial button 55, and in response to the click operation of the user on the trial button 55, the electronic device 900 enters the guide flow provided in the foregoing embodiments. It should be noted that after the user enters the guide flow, whether the user interrupts the guide flow or not, when the virtual assistant is woken up again subsequently, the electronic device 900 no longer displays the first navigation page 53.

[0243] The user can click the quit button 56, and in response to the click operation of the user on the quit button 56, the electronic device 900 closes the first navigation page 53 and displays the first window corresponding to the virtual assistant. It should be noted that the quit button 56 has different display forms, for example, the text description corresponding to the quit button 56 can be “quit” or “I know”, so as to be adaptable to the language habits of the user. Although different language description methods can be used, the quit button 56 with different language description methods all perform the function of closing the first navigation page 53.

[0244] If the electronic device 900 closes the first navigation page 53, it indicates that the user does not enter the guide flow based on actual operation, and then the electronic device 900 can display the first navigation page 53 again in the process of waking up the virtual assistant subsequently by the user. The number of times of displaying the first navigation page 53 can be set in the embodiments of the present application, for example, the first navigation page 53 can be displayed only once within a day, and the total number of times of displaying the first navigation page 53 can be three. The display frequency of the first navigation page 53 can be set according to actual conditions, which is not limited in the embodiments of the present application.

[0245] In an implementation manner, the electronic device 900 can skip the step of displaying the first navigation page 53 when the virtual assistant is woken up for the first time based on system settings.

[0246] It should be noted that in addition to the manners of waking up the virtual assistant based on voice and based on a Bluetooth key, the manners of waking up the virtual assistant for the first time of the rest virtual assistants can be referred to the manner of waking up the virtual assistant for the first time based on the power key 52, which is not described herein again.

[0247] In this way, the user can use the virtual assistant after waking up the virtual assistant for the first time and completing the guide.

[0248] In addition, in order to further ensure that the user understands the use manner of the virtual assistant, in the process of using the virtual assistant by the user, in response to the user triggering the window state switching of the virtual assistant for the first time, the electronic device 900 displays a first navigation prompt box to guide the user to move the first window.

[0249] FIG. 11 is a schematic diagram of a scenario of switching window state of triggering a virtual assistant for the first time according to an embodiment of the present application.

[0250] As shown in FIG. 11, the user can wake up the virtual assistant on the home screen page 57 of the electronic device 900 based on the power key 52 to display a first window 58 corresponding to the virtual assistant and at least one first control 59 adjacent to the first window 58, the first control 59 including a service recommendation matching the home screen page 57, for example, “organize the desktop”, “change the wallpaper”, etc.

[0251] The first window 58 is displayed by default as a first state window 60 when it is woken up. The first state window 60 is described in detail in subsequent embodiments of the present application.

[0252] The first state window 60 includes a navigation bar 61, which can be displayed at the top of the first state window 60.

[0253] The user can input text content in the first state window 60, and in response to the user inputting the text content, the electronic device 900 displays a second state window 62 so that the second state window 62 can accommodate the interaction content between the user and the virtual assistant. The second state window 62 is described in detail in subsequent embodiments of the present application.

[0254] In the case where the first state window 60 is switched to the second state window 62, the navigation bar 61 can be displayed at the top of the second state window 62.

[0255] In response to the electronic device 900 displaying the second state window 62 for the first time, the electronic device 900 displays a first navigation prompt box 63 in a region adjacent to the navigation bar 61 in the second state window 62 to guide the user to move the second state window 62.

[0256] For example, the prompt content in the first navigation prompt box 63 can be “try pressing and dragging the top bar to move the window position.”

[0257] The first navigation prompt box 63 further includes an exit button 64. The user can click the exit button 64, and in response to the user clicking the exit button 64, the electronic device 900 stops displaying the first navigation prompt box 63.

[0258] The user can perform a long press and drag operation on the navigation bar 61 according to the prompt content in the first navigation prompt box 63, and in response to the user performing the long press and drag operation on the navigation bar 61, the electronic device 900 moves the second state window 62 in the home screen page 57 and stops displaying the first navigation prompt box 63.

[0259] The second state window 62 further includes an exit button 65. The user can click the exit button 65, and in response to the user's click operation on the exit button 65, the electronic device 900 ends the session of the virtual assistant and stops displaying the first navigation prompt box 63.

[0260] The user can also end the session of the virtual assistant based on the manner of the gesture operation, and in response to the user's gesture operation, the electronic device 900 ends the session of the virtual assistant and stops displaying the first navigation prompt box 63.

[0261] Embodiments of the present application do not limit the manner of stopping displaying the first navigation prompt box 63.

[0262] Embodiments of the present application can set the number of times of displaying the first navigation prompt box 63, for example, set the number of times of displaying the first navigation prompt box 63 to be displayed only once when the second state window 62 is triggered for the first time. The display frequency of the first navigation prompt box 63 can be set according to actual conditions, and embodiments of the present application do not limit this.

[0263] After the electronic device 900 is powered on, the default setting mode is usually adopted for the virtual assistant, and the electronic device 900 can also change the related setting mode of the virtual assistant in the setting application software.

[0264] The setting mode of the virtual assistant is described in detail below.

[0265] FIG. 12 is a first scenario diagram of the setting mode of the virtual assistant according to an embodiment of the present application.

[0266] As shown in FIG. 12, the electronic device 900 can display a setting application program 66 in the home screen page 57. The user can perform a click operation on the setting application program 66, and in response to the user's click operation on the setting application program 66, the electronic device 900 displays a first setting page 67, and the first setting page 67 can include a virtual assistant option 68.

[0267] The user can perform a click operation on the virtual assistant option 68, and in response to the user's click operation on the virtual assistant option 68, the electronic device 900 enters a second setting page 69, and the second setting page 69 can include a virtual assistant setting option 70.

[0268] The user can perform a click operation on the virtual assistant setting option 70, and in response to the user's click operation on the virtual assistant setting option 70, the electronic device 900 enters a third setting page 71, and the third setting page 71 can include a system navigation wake-up option 72.

[0269] The user can click the system navigation wake-up option 72, and in response to the user's click operation on the system navigation wake-up option 72, the electronic device 900 enters a fourth setting page 73, which can include a system navigation wake-up control 74. The system navigation wake-up control 74 can be in a default on state, so that the virtual assistant can be woken up by the navigation bar 61 or the home button in the navigation bar on any page of the electronic device 900.

[0270] If the system navigation wake-up control 74 is set to the off state, the electronic device 900 does not display the navigation bar 61.

[0271] Since the wake-up mode of the virtual assistant is associated with different system navigation modes, the related setting item of the virtual assistant is also included in the setting page of the system navigation mode.

[0272] FIG. 13 is a second scenario diagram of a setting mode of a virtual assistant according to an embodiment of the present application.

[0273] As shown in FIG. 13, the electronic device 900 can display a setting application 66 in the home screen page 57. The user can click the setting application 66, and in response to the user's click operation on the setting application 66, the electronic device 900 displays a first setting page 67, which can also include a system and update option 75.

[0274] The user can click the system and update option 75, and in response to the user's click operation on the system and update option 75, the electronic device 900 enters a fifth setting page 76, which can include a system navigation mode setting option 77.

[0275] The user can click the system navigation mode setting option 77, and in response to the user's click operation on the system navigation mode setting option 77, the electronic device 900 enters a sixth setting page 78, which can include a gesture navigation option 79 and a three-key navigation option 80.

[0276] It should be noted that the three-key navigation is a navigation bar that provides the user with operation buttons, which usually includes a back button, a home button, and a multi-function button. The present application only exemplarily describes the navigation bar with three-key navigation, and in fact, the navigation bar can also include more buttons, which are not limited by the present application.

[0277] The user can perform a click operation on the gesture navigation option 79. In response to the user's click operation on the gesture navigation option 79, the electronic device 900 enters a gesture navigation page 81. The gesture navigation page 81 can include a display prompt bar control 82 and a system navigation wake-up control 83. Both the display prompt bar control 82 and the system navigation wake-up control 83 can be in a default on state. Thus, on any page of the electronic device 900, the virtual assistant can be woken up through the navigation bar 61.

[0278] The user can perform a click operation on the three-key navigation option 80. In response to the user's click operation on the three-key navigation option 80, the electronic device 900 enters a three-key navigation page 84. The three-key navigation page 84 can include a system navigation wake-up control 85. The system navigation wake-up control 85 can be in a default on state. Thus, on any page of the electronic device 900, the virtual assistant can be woken up through the home screen button of the navigation bar.

[0279] The use process of the virtual assistant is further described below.

[0280] Since the virtual assistant in the embodiments of the present application can be used in cooperation with an application program, the use process of the virtual assistant in an application program is exemplarily described in the embodiments of the present application.

[0281] FIG. 14 is a schematic diagram of a first wake-up mode of the virtual assistant in an application program according to an embodiment of the present application.

[0282] As shown in FIG. 14, the electronic device 900 can display a program icon 86 of a first application program in the home screen page 57. The first application program can be, for example, an email application program.

[0283] The first application program can also be a chat application program or other types of application programs. The specific type of the first application program is not limited in the embodiments of the present application.

[0284] The user can perform a click operation on the program icon 86. In response to the user's click operation on the program icon 86, the electronic device 900 displays a first page 87.

[0285] The gesture navigation of the electronic device 900 is exemplarily described. In the case where the electronic device 900 adopts gesture navigation, the electronic device 900 displays the navigation bar 61 at the bottom of the display screen.

[0286] The user can perform a long press operation on the navigation bar 61. In response to the long press operation of the user on the navigation bar 61, the electronic device 900 wakes up the virtual assistant on the first page 87 to display a first status window 60 corresponding to the virtual assistant and at least one first control 59 adjacent to the first status window 60, the first control 59 including a service recommendation matching the first page 87.

[0287] It should be noted that the user can also perform other types of operations on the navigation bar 61 to wake up the virtual assistant, for example, a sliding operation, a multiple click operation, a custom gesture operation, and the like. The present embodiment only exemplarily describes the long press operation on the navigation bar 61, and the present embodiment does not limit the type of gesture operation for waking up the virtual assistant.

[0288] In an implementation manner, in order to improve the user experience, during the long press operation of the user on the navigation bar 61, the navigation bar 61 can be changed in a dynamic effect to indicate that the user is currently performing a long press operation.

[0289] FIG. 15 is a schematic diagram of a dynamic effect change manner of a navigation bar according to an embodiment of the present application.

[0290] As shown in FIG. 15, the initial display specification of the navigation bar 61 can be 4x120dp.

[0291] The density-independent pixels (dp) is a pixel unit used to abstract and standardize different screen densities.

[0292] During the long press operation of the user on the navigation bar 61, the electronic device 900 can change the display specification of the navigation bar 61 from 4x120dp to 4x140dp, and display a first dynamic effect on the navigation bar 61 for a preset dynamic effect duration. The preset dynamic effect duration can be, for example, 1 second, and the first dynamic effect can be a flashing dynamic effect. The present embodiment does not limit the specific duration of the preset dynamic effect duration and the dynamic effect type of the first dynamic effect.

[0293] Further, during the long press operation of the user on the navigation bar 61, the electronic device 900 can provide vibration feedback through a motor to improve the user's pressing experience.

[0294] The following exemplarily describes other wake-up manners of the virtual assistant.

[0295] FIG. 16 is a schematic diagram of a second wake-up manner of a virtual assistant in an application according to an embodiment of the present application.

[0296] As shown in FIG. 16, an example is illustrated in which the electronic device 900 adopts a three-key navigation mode. In the case where the electronic device 900 adopts the three-key navigation mode, the electronic device 900 displays the navigation bar 88 at the bottom of the display screen thereof, and displays the home screen button 89 in the navigation bar 88.

[0297] The user can perform a long press operation on the home screen button 89. In response to the long press operation of the user on the home screen button 89, the electronic device 900 awakens the virtual assistant on the first page 87 to display the first status window 60 corresponding to the virtual assistant, and at least one first control 59 adjacent to the first status window 60, the first control 59 including a service recommendation matching the first page 87.

[0298] It should be noted that the user can also perform other types of operations on the home screen button 89 to awaken the virtual assistant, for example, a sliding operation, a multiple-click operation, a custom gesture operation, etc. The present embodiment is only illustratively described in the manner of performing a long press operation on the home screen button 89, and the present embodiment does not limit the type of gesture operation for awakening the virtual assistant.

[0299] In an implementation manner, in order to improve the user experience, during the long press operation of the user on the home screen button 89, the navigation bar 88 and the home screen button 89 can be caused to change in animation effect to indicate that the user is currently performing a long press operation. The present embodiment does not limit the specific manner in which the home screen button 89 changes in animation effect.

[0300] Since the electronic device 900 displays both the navigation bar 88 and the first status window 60 on the first page 87, the first status window 60 can avoid the navigation bar 88, for example, by being displayed in the top area of the navigation bar 88.

[0301] It should be noted that in the subsequent use of the virtual assistant, the first status window 60 can be switched to a window in another state, and the first window 58 in any state can avoid the navigation bar 88.

[0302] FIG. 17 is a schematic diagram of a third awakening manner of the virtual assistant in an application according to an embodiment of the present application.

[0303] As shown in FIG. 17, whether the electronic device 900 adopts a gesture navigation mode or a three-key navigation mode, the electronic device 900 can awaken the virtual assistant based on a key. An example is illustratively described in which the key is the power key 52.

[0304] The user can perform a long press operation on the power key 52. In response to the long press operation of the user on the power key 52, the electronic device 900 wakes up the virtual assistant on the first page 87 to display a first status window 60 corresponding to the virtual assistant and at least one first control 59 adjacent to the first status window 60, the first control 59 including a service recommendation matching the first page 87.

[0305] It should be noted that the user can also perform other types of operations on the power key 52 to wake up the virtual assistant, for example, multiple click operations, etc. The present embodiment only exemplarily describes the long press operation on the power key 52, and the present embodiment does not limit the operation type for waking up the virtual assistant.

[0306] FIG. 18 is a schematic diagram of a fourth wake-up mode of the virtual assistant in an application according to an embodiment of the present application.

[0307] As shown in FIG. 18, a gesture navigation mode of the electronic device 900 is exemplarily described. In the case of the gesture navigation mode of the electronic device 900, the electronic device 900 displays a navigation bar 61 at the bottom of the display screen.

[0308] The user can wake up the virtual assistant on the first page 87 by dragging the target page content 90 of the first page 87 to the navigation bar 61.

[0309] The user can perform a selection operation on the target page content 90 of the first page 87. In response to the selection operation on the target page content 90, the electronic device 900 generates a first selected area 91 at the position of the target page content 90. The first selected area 91 can be highlighted.

[0310] It should be noted that the target page content 90 in the present embodiment can be at least one of a picture, a text, a document, and a video, and the present embodiment does not limit the specific type of the target page content 90.

[0311] The user can perform a long press operation on the first selected area 91. In response to the long press operation of the user on the first selected area 91, the electronic device 900 generates a first moving area 92 for copying and moving the target page content 90.

[0312] It should be noted that when the long press operation is performed on the first selected area 91, the navigation bar 61 can change in dynamic effect to indicate that the user can wake up the virtual assistant at the corresponding position to provide the corresponding service.

[0313] In an implementation manner, during the long-press operation of the user on the first selected region 91, the electronic device 900 can change the display specification of the navigation bar 61 from 4x120dp to 4x140dp, and display the first dynamic effect on the navigation bar 61 at a preset dynamic effect frequency. For example, the preset dynamic effect frequency can be 3 times, and the first dynamic effect can be a flashing dynamic effect. The specific duration of the preset dynamic effect duration and the dynamic effect type of the first dynamic effect are not limited in the embodiments of the present application.

[0314] The user can drag the first moving region 92 to achieve the user's demand. It should be noted that the electronic device 900 is provided with a plurality of services. For example, an intent recognition service. The intent recognition service allows the user to realize information transmission and operation across applications and devices through the drag operation. The user can trigger the intent recognition service by dragging the first moving region 92 to the screen edge.

[0315] That is, during the dragging of the first moving region 92 by the user, the electronic device 900 cannot determine that the user's intention is to call the virtual assistant or to call the intent recognition service. Therefore, the electronic device 900 needs to identify the user's intention to call the corresponding service.

[0316] The electronic device 900 can identify the user's intention based on the region corresponding to the drag operation to call the corresponding service.

[0317] FIG. 19 is a schematic diagram of a scenario in which the electronic device identifies the user's intention according to an embodiment of the present application.

[0318] As shown in FIGS. 18 and 19, the electronic device 900 can be provided with a first preset region 95. The first preset region 95 can be a region extending a certain height from the bottom edge of the display screen, which can account for 10% of the total area of the display screen, for example. The position and area of the first preset region are not limited in the embodiments of the present application.

[0319] During the dragging of the first moving region 92 by the user, in response to the drag operation of the user on the first moving region 92, when the first moving region 92 does not reach the first preset region 95, the electronic device 900 displays a first transition window 93 at the position of the navigation bar 61 and displays a first light effect at the screen edge 94.

[0320] The first transition window 93 can be in the form of a capsule, and the first light effect can have a weak light intensity. The display form of the first transition window 93 and the light effect intensity of the first light effect are not limited in the embodiments of the present application.

[0321] In this way, the first transition window 93 and the first light effect of the screen edge 94 guide the user at the same time, so that the user can select to perform the corresponding function.

[0322] In this case, the user can drag the first moving area 92 to the first transition window 93, or drag the first moving area 92 to the screen edge 94.

[0323] That is, based on the setting of the first preset area 95, it is ensured that the electronic device 900 will not mistakenly invoke the virtual assistant before the first moving area 92 reaches the first preset area 95, avoiding the mismatch between the services provided by the virtual assistant and the actual intention of the user. It should be noted that the manner in which the electronic device 900 determines that the first moving area 92 has not reached the first preset area 95 can be based on detecting that the first moving area 92 moves on other preset areas.

[0324] During the process in which the user further drags the first moving area 92 to the first transition window 93, in the case where the first moving area 92 reaches the first preset area 95, the electronic device 900 displays a second transition window 96 at the location of the first transition window 93, and stops displaying the first light effect at the screen edge 94.

[0325] In this case, the electronic device 900 defaults that the user needs to invoke the virtual assistant, and closes the intention recognition service, so the user cannot drag the first moving area 92 to the screen edge 94. In this way, the accuracy of the electronic device 900 in recognizing the intention of the user is ensured.

[0326] The second transition window 96 can be in a capsule form, and the capsule form of the second transition window 96 can be different from the capsule form of the first transition window 93. For example, the second transition window 96 can increase a second dynamic effect on the window edge on the basis of the capsule form of the first transition window 93. The second dynamic effect can be at least one of a color flashing dynamic effect and a halo dynamic effect, and the embodiments of the present application do not limit the display form of the second transition window 96 and the type of the second dynamic effect.

[0327] In the case where the first moving area 92 reaches the second transition window 96, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first state window 60 corresponding to the virtual assistant. (The process of displaying the first state window 60 is not shown in the figure). Further, the user drags the first moving area 92 to the first state window 60, and in response to the operation of the user dragging the first moving area 92 into the first state window 60, the electronic device 900 displays the second state window 62 and displays the target page content 90 in the second state window 62.

[0328] In this embodiment, the first transition window 93 and the second transition window 96 are provided, which not only ensures that the electronic device 900 can accurately identify the user's intention, but also forms a coherent visual experience between the first transition window 93, the second transition window 96, the first state window 60, and the second state window 62, so as to improve the user's visual experience. The electronic device 900 avoids the screen hot area (for example, the screen edge 94 corresponding to the intention recognition service) based on the setting of the first preset area 95.

[0329] The electronic device 900 can also identify the user's intention based on the displacement corresponding to the drag operation to call the corresponding service. This manner can be implemented by referring to the foregoing embodiments. For example, in the process that the user drags the first moving area 92, in response to the drag operation of the user on the first moving area 92, the electronic device 900 displays the first transition window 93 at the position where the navigation bar 61 is located and displays the first light effect at the screen edge 94 without the first moving area 92 reaching the first preset displacement. Details are not described herein again.

[0330] The electronic device 900 can also use other manners of identifying the user's intention, and the specific manner of identifying the user's intention by the electronic device 900 is not limited in the embodiments of the present application.

[0331] In an implementation manner, the user can perform the operation of multiple times of selecting and dragging different target page contents 90 in the first page 87. If the user does not wake up the virtual assistant within a preset number of times of dragging, it is indicated that the operation of selecting and dragging by the user is for the intention recognition service. Thus, after the preset number of times of dragging is reached, if the user performs the operation of selecting and dragging again, the electronic device stops displaying the first transition window 93 and only displays the first light effect at the screen edge 94.

[0332] FIG. 20 is a schematic diagram of a fifth wake-up manner of a virtual assistant in an application program according to an embodiment of the present application.

[0333] As shown in FIG. 20, the electronic device 900 is exemplarily described in the manner of three-key navigation. In the case that the electronic device 900 uses the three-key navigation, the electronic device 900 displays the navigation bar 88 at the bottom of the display screen and displays the home screen button 89 in the navigation bar 88.

[0334] The user can wake up the virtual assistant on the first page 87 by dragging the target page content 90 of the first page 87 to the home screen button 89.

[0335] The specific process that the user drags the target page content 90 of the first page 87 to the home screen button 89 can be referred to the foregoing embodiments.

[0336] In this embodiment, when the electronic device 900 displays the first transition window 93 and the second transition window 96, the first transition window 93 and the second transition window 96 can be overlaid on the navigation bar 88, or the navigation bar 88 can be avoided.

[0337] When the electronic device 900 displays the first window 58, the navigation bar 88 can be avoided, for example, the first window 58 is displayed in the top area of the navigation bar 88. The display position of the first window 58 is not limited in the embodiments of the present application.

[0338] Among the fourth and fifth wake-up modes of the virtual assistant in the application program, the way in which the electronic device 900 identifies the user's intention is involved. It should be noted that the way in which the electronic device 900 identifies the user's intention can be applicable to the horizontal screen mode and the vertical screen mode of the electronic device 900, or can be applicable to the unfolded state and the folded state of the foldable electronic device when the electronic device 900 is a foldable electronic device. The priority setting of the virtual assistant and the intention recognition service in different use modes of the electronic device 900 is not limited in the embodiments of the present application.

[0339] The electronic device 900 can set different priorities for the virtual assistant and the intention recognition service in different use modes, for example, in the horizontal screen mode, the response priority of the navigation bar 88 can be lower than that of the intention recognition service. Specifically, in the horizontal screen mode, the navigation bar 88 and the screen edge 94 can be located on the right side of the display screen. At this time, the screen edge 94 is the bottom edge of the display screen in the vertical screen mode. The screen edge 94 responds to the user's drag operation first to make the electronic device 900 execute the intention recognition service. The priority setting mode of the electronic device 900 in different use modes is not limited in the embodiments of the present application.

[0340] The electronic device 900 can have more ways to wake up the virtual assistant, which are not limited in the embodiments of the present application.

[0341] When the electronic device 900 wakes up the virtual assistant, the first window 58 corresponding to the virtual assistant is displayed. The first window 58 has different display states, which are described in detail below.

[0342] FIG. 21 is a schematic diagram of the display state of the first window according to an embodiment of the present application.

[0343] The first window 58 includes a first state window 60, a second state window 62, and a third state window 97.

[0344] As shown in A of FIG. 21, the first state window 60 can include a first interaction area 98, which can include an input bar 99, a first assistant icon 100, a multifunction button 101, and a voice input button 102. The first interaction area 98 is used to receive the input operation of the user.

[0345] The input bar 99 can be used for the user to perform an input operation, the first assistant icon 100 can prompt the user that the current window is used for a conversation with a virtual assistant, the multi-functional button 101 can provide a third-party service, and the voice input button 102 can enable the user to perform a voice input operation.

[0346] It should be noted that other types of window elements can also be arranged in the first state window 60 according to actual conditions to implement different functions, and the specific content arranged in the first state window 60 is not limited in the embodiments of the present application.

[0347] The first state window 60 can further include a first window dynamic effect 103. The first window dynamic effect 103 can be a colored frame surrounding the first interaction area 98, or can be other types of dynamic effects, and the type of the first window dynamic effect 103 is not limited in the embodiments of the present application.

[0348] The first state window 60 can have a first height H1. The first height H1 usually accounts for a small proportion of the total height of the display screen, because the first state window 60 is usually a window corresponding to the case that the virtual assistant is initially woken up and the user does not input interaction content.

[0349] In the first state window 60, the navigation bar 61 is displayed in the top area of the first state window 60. In the case that the electronic device 900 displays the first state window 60, the first control 59 can be displayed in the outer peripheral adjacent area of the first state window 60.

[0350] The second state window 62 can include a first sub-window 104 and a second sub-window 105.

[0351] As shown in B of FIG. 21, the first sub-window 104 includes the first interaction area 98 and a first panel 106, and the first panel 106 is displayed in a part of the display screen of the electronic device 900.

[0352] It should be noted that the first interaction area 98 in the first sub-window 104 can be the same as the first interaction area 98 in the first state window 60. This will not be repeated in subsequent embodiments of the present application.

[0353] The first sub-window 104 can further include a second window dynamic effect 107. The second window dynamic effect 107 can be a colored frame with a flowing light effect surrounding the first panel 106, or can be a colored frame with a flowing light effect surrounding the first interaction area 98, or can be other types of dynamic effects, and the type of the second window dynamic effect 107 is not limited in the embodiments of the present application.

[0354] It should be noted that, in the process of switching the first sub-window 104 to the first window 58 in other states, the second window effect 107 can switch between different effects contained therein, for example, from a colored frame with flowing light effect surrounding the first interactive area 98 to a colored frame with flowing light effect surrounding the first panel 106.

[0355] The first sub-window 104 can have a second height H2, which can be greater than the first height H1, for example, can account for 40%-50% of the total height of the display screen, and the specific height of the second height H2 is not limited in the embodiments of the present application. Based on the second height H2, the first sub-window 104 can accommodate the interaction content between the virtual assistant and the user through the first panel 106.

[0356] In the first sub-window 104, the navigation bar 61 is displayed at the top area of the first panel 106, and the first control 59 can be displayed in the first panel 106 and in the outer peripheral adjacent area of the first interactive area 98.

[0357] It should be noted that, for the first window 58 in different window states, the first control 59 can have different display forms, for example, the first control 59 in the first sub-window 104 can be provided with an effect to improve the user's visual experience. The effect can be a blue background, for example. The display form of the first control 59 is not limited in the embodiments of the present application.

[0358] As shown in C of FIG. 21, the second sub-window 105 includes the first interactive area 98 and the second panel 108, which is displayed in a part of the display screen of the electronic device 900.

[0359] The second sub-window 105 can further include a third window effect 109, which can be a colored frame with flowing light effect surrounding the second panel 108, or a colored frame with flowing light effect surrounding the first interactive area 98, or other types of effects, and the type of the third window effect 109 is not limited in the embodiments of the present application.

[0360] It should be noted that, in the process of switching the second sub-window 105 to the first window 58 in other states, the third window effect 109 can switch between different effects contained therein, for example, from a colored frame with flowing light effect surrounding the first interactive area 98 to a colored frame with flowing light effect surrounding the second panel 108.

[0361] The second sub-window 105 can have a third height H3, which can be greater than the second height H2, and the third height H3 can account for 60%-80% of the total height of the display screen, for example, and embodiments of the present application do not limit the specific height of the third height H3. Based on the third height H3, the second sub-window 105 can accommodate more interaction content between the virtual assistant and the user through the second panel 108.

[0362] As shown in D in FIG. 21, the third state window 97 can be displayed in the form of a third panel 110 or in the form of an in-application page corresponding to the virtual assistant.

[0363] A page is generally the entire area of a user interface. A panel is generally a specific area in a user interface, and the panel can be floating or fixed on the page. Embodiments of the present application do not limit the specific display form of the third state window 97. Based on the third state window 97, the user can operate within the application corresponding to the virtual assistant.

[0364] Regardless of whether the third state window 97 is displayed in the form of the third panel 110 or in the form of an in-application page corresponding to the virtual assistant, the third state window 97 can cover the full-screen area.

[0365] Embodiments of the present application only exemplarily illustrate the form in which the third state window 97 is displayed in the third panel 110.

[0366] The third panel 110 is displayed in the full-screen area corresponding to at least one display screen, and the third panel 110 can not only be used to accommodate interaction content between the virtual assistant and the user, but also provide more functions. Specifically, the third panel 110 can include multiple sub-panels, such as a virtual assistant panel, an agent store panel, and a recent conversation panel.

[0367] The virtual assistant panel is a panel for the user to interact with the virtual assistant, and is the default sub-panel in the third panel 110. The first interaction area 98 can be displayed on the virtual assistant panel.

[0368] The agent store panel can provide various agent applications, and different agent applications can be suitable for different use scenarios, such as customer service, education, entertainment, medical treatment, etc. The user can select a corresponding agent application in the agent store panel and apply it in the virtual assistant to meet the user's use requirements.

[0369] The recent conversation panel can store historical conversation content.

[0370] The third panel 110 can be provided with a switching button 111, which can include a virtual assistant button, an agent store button, and a recent conversation button. The virtual assistant button corresponds to the virtual assistant panel, the agent store button corresponds to the agent store panel, and the recent conversation button corresponds to the recent conversation panel.

[0371] The user can perform a click operation on the switching button 111, and in response to the click operation of the user, the electronic device 900 can display the sub-panel corresponding to the switching button 111 to perform a corresponding function on the sub-panel.

[0372] It should be noted here that in the case where the third panel 110 is the agent store panel or the recent conversation panel, since no interaction operation is required on these panels, the electronic device 900 does not display the first interaction area 98 on the agent store panel and the recent conversation panel, but displays the navigation bar 61. That is, on these panels, the navigation bar 61 and the first interaction area 98 cannot be displayed at the same time. In fact, in any embodiment of the present application, the navigation bar 61 cannot be displayed at the same time as the first interaction area 98.

[0373] The third state window 97 in the following embodiments of the present application refers to a window displaying the virtual assistant panel.

[0374] The third state window 97 can also include a fourth window dynamic effect 112, which can be a colored frame with a flowing light effect surrounding the third panel 110, or a colored frame with a flowing light effect surrounding the first interaction area 98, or other types of dynamic effects, and the embodiments of the present application do not limit the type of the fourth window dynamic effect 112.

[0375] It should be noted here that in the process of switching the third state window 97 to the first window 58 of other states, the fourth window dynamic effect 112 can switch between different dynamic effects contained therein, for example, from a colored frame with a flowing light effect surrounding the first interaction area 98 to a colored frame with a flowing light effect surrounding the third panel 110.

[0376] The third state window 97 can have a fourth height H4, which can account for 90%-100% of the total height of the display screen, and the specific height of the fourth height H4 is not limited in the embodiments of the present application. Based on the fourth height H4, the third state window 97 can accommodate more interaction content between the virtual assistant and the user through the third panel 110.

[0377] In order to conform to the user's operation habits, the navigation bar 61 can not be displayed in the third state window 97, so that the third state window 97 is closer to the display mode of the page.

[0378] In the third state window 97, the first control 59 can be displayed in the virtual assistant panel and in the peripheral adjacent area of the first interaction area 98.

[0379] The first window 58 has different input states based on different ways of waking up the virtual assistant by the electronic device 900.

[0380] The first window 58 has different input states based on different ways of waking up the virtual assistant by the electronic device 900.

[0381] The first state window 60 is exemplarily described for different input states.

[0382] As shown in A of FIG. 22, in a case where the user adopts the first wake-up manner, the second wake-up manner, the fourth wake-up manner, and the fifth wake-up manner (i.e., the manner of waking up the virtual assistant based on the long-press operation on the navigation bar 61, the long-press operation on the home button 89, or the drag operation) in the foregoing embodiments, the electronic device 900 displays the first state window 60 with a first input state in response to the wake-up operation of the user, and the first input state is a state in which no voice input is currently received.

[0383] In the first input state, the user cannot directly perform voice input on the electronic device 900, but needs to click the voice input button 102 in the first state window 60 to switch to a second input state and then perform voice input. It should be noted that the voice input button 102 can be set to support the click operation or the long-press operation in the embodiments of the present application, and the specific setting can be performed according to the use habit of the user, which is not limited in the embodiments of the present application.

[0384] As shown in B of FIG. 22, in a case where the user adopts the third wake-up manner (i.e., the manner of waking up the virtual assistant based on the long-press operation on the power button 52) in the foregoing embodiments, the electronic device 900 displays the first state window 60 with a second input state in response to the wake-up operation of the user, and the second input state is a state in which voice input is currently received.

[0385] In the second input state, the user directly performs voice input on the electronic device 900.

[0386] It should be noted that the first window dynamic effect 103 corresponding to the first state window 60 can be different in different input states. For example, in the first input state, the first window dynamic effect 103 can be a colored frame surrounding the first interaction area 98, in the second input state, the first window dynamic effect 103 can be a halo dynamic effect or a flowing light spot dynamic effect, and in the second input state, the voice input button 102 can be displayed in a highlighted state, and the first assistant icon 100 can be displayed in a tilt dynamic effect. The dynamic effect that the first assistant icon 100 can be displayed in is not limited in the embodiments of the present application.

[0387] The application is further described by the user waking up the virtual assistant in the first wake-up mode.

[0388] As further shown in FIG. 14, in the first wake-up mode, the user performs a long press operation on the navigation bar 61 on the first page 87. In response to the long press operation of the user on the navigation bar 61, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first status window 60 corresponding to the virtual assistant and at least one first control 59 adjacent to the first status window 60. The first control 59 includes a service recommendation matching the first page 87.

[0389] The first control 59 is a carrier for inferring a user's scenario intention based on screen understanding and providing services. It can provide dynamic service recommendations according to content understanding.

[0390] The first control 59 can provide service recommendations supported by the virtual assistant and service recommendations supported by third-party services. For example, the service recommendation supported by the virtual assistant can be a service recommendation for object recognition, and the service recommendation supported by the third-party service can be a service recommendation for jumping to another application to realize a corresponding function, such as a service recommendation for jumping to a note application to save notes. The application does not limit the specific types of service recommendations supported by the first control 59.

[0391] The following exemplary describes some service recommendations supported by the first control 59.

[0392] In the case where the first page 87 includes text content and the text content includes an address, if the address includes at least one of an entity extraction address, a store name, and a scenic spot name, the first control 59 can provide service recommendations for navigation, taking a car, viewing the address, viewing the distance, and collecting, etc.

[0393] For example, the recommended content of the service recommendation includes "navigate to here", "take a car to here", "view address", "distance", and "collection", etc. The application does not limit the recommended types and contents of the service recommendation.

[0394] In the case where the first page 87 includes text content and the text content includes a non-native language, the first control 59 can provide a service recommendation for translation.

[0395] For example, the recommended content of the service recommendation includes "translation".

[0396] In a case where the first page 87 includes picture content, and the picture content includes a form filling interface, if the electronic device 900 detects that the picture is a form filling interface, and the related subject matches the business logic of form filling, the first control 59 can provide a service recommendation for form filling.

[0397] For example, the recommended content of the service recommendation includes "form filling".

[0398] In a case where the first page 87 includes picture content, and the picture content includes a chat interface, if the electronic device 900 determines that the picture is a chat record based on label recognition, the first control 59 can provide a service recommendation for privacy coding.

[0399] For example, the recommended content of the service recommendation includes "privacy coding".

[0400] In a case where the first page 87 includes picture content, and the picture content includes a product, if the electronic device 900 determines that the picture is a product picture based on label recognition, the first control 59 can provide a service recommendation for shopping, collecting.

[0401] For example, the recommended content of the service recommendation includes "shopping", "collecting".

[0402] In a case where the first page 87 includes picture content, and the picture content includes a person, if the electronic device 900 detects multiple persons, the first control 59 can provide a service recommendation for automatic elimination.

[0403] For example, the recommended content of the service recommendation includes "eliminate passersby".

[0404] In a case where the first page 87 includes picture content, and the picture content includes a person, if the electronic device 900 detects a person face, the first control 59 can provide a service recommendation for automatic beautification.

[0405] For example, the recommended content of the service recommendation includes "one-key skin beautification".

[0406] In a case where the first page 87 includes picture content, and the picture content includes a person, if the electronic device 900 detects a person face, and the face area is greater than 100 pixels x 100 pixels, the first control 59 can provide a service recommendation for face changing detection.

[0407] For example, the recommended content of the service recommendation includes "AI face changing detection".

[0408] In a case where the first page 87 includes picture content, and the picture content includes a person, if the electronic device 900 detects a person face, and the face is a front face, the first control 59 can provide a service recommendation for generating an ID photo.

[0409] For example, the recommended content of the service recommendation includes “generate ID photo”.

[0410] In a case where the first page 87 includes picture content, and the picture content is a photo taken by a camera, the first control 59 can provide a service recommendation for adding a time and location sticker.

[0411] For example, the recommended content of the service recommendation includes “add a time and location sticker”.

[0412] In a case where the first page 87 includes picture content, and the picture content includes a subject, the first control 59 can provide a service recommendation for cutting out, extracting card information.

[0413] For example, the recommended content of the service recommendation includes “automatically cut out”, “extract card information to notes”, “save card information to personal information”, “copy address”, and “copy ID number”.

[0414] In a case where the first page 87 includes a desktop customization page, the first control 59 can provide a service recommendation for customizing the desktop.

[0415] For example, the recommended content of the service recommendation includes “customize the desktop”.

[0416] In a case where the first page 87 includes a video playing page or an audio playing page, the first control 59 can provide a service recommendation for translation.

[0417] For example, the recommended content of the service recommendation includes “AI caption”.

[0418] Based on different service recommendations, the first control 59 can have different display forms.

[0419] FIG. 23 is a schematic diagram of a display mode of the first control according to an embodiment of the present application.

[0420] As shown in A of FIG. 23, the first control 59 can be displayed based on a pure text style.

[0421] As shown in B of FIG. 23, the first control 59 can be displayed based on a star mark and a text style. In this way, the recommendation intention of the first control 59 can be enhanced.

[0422] As shown in C of FIG. 23, the first control 59 can be displayed based on an entity button 113 and a text style. In this way, based on the entity button 113, the electronic device 900 can jump to other services, such as a positioning service, and based on the entity button 113, the electronic device 900 can also implement related functions such as saving and printing. The types of functions that can be implemented by the entity button 113 are not limited in the present application.

[0423] And, in the same first control 59, a plurality of entity buttons 113 can be arranged, so that the same first control 59 implements a plurality of different functions, the number of functions that the first control 59 can support can be 4 for example, the number of functions that the first control 59 can support is not limited in the embodiment of the application.

[0424] As shown in D in FIG. 23, the first control 59 can be displayed based on the application icon 114 and the text style. In this way, based on the first control 59, the electronic device 900 can jump to the application program corresponding to the application icon 114.

[0425] The display form of the first control 59 is not limited in the embodiment of the application.

[0426] Further as shown in FIG. 14, a plurality of first controls 59 can be arranged in the outer peripheral adjacent area of the first status window 60, for example, 8 first controls 59 are arranged, wherein a part of the first controls 59 can be displayed in the outer peripheral adjacent area of the first status window 60, for example, 2 first controls 59 are displayed in the outer peripheral adjacent area of the first status window 60, and another part of the first controls 59 can be hidden in the outer peripheral adjacent area of the first status window 60, for example, 6 first controls 59 are hidden in the outer peripheral adjacent area of the first status window 60.

[0427] The user can perform a sliding operation in the outer peripheral adjacent area of the first status window 60, and in response to the sliding operation of the user, the electronic device 900 can alternately display the first controls 59.

[0428] The user can perform a clicking operation on the first control 59, and in response to the clicking operation of the user on the first control 59, the electronic device 900 can execute the function corresponding to the service recommendation in the first control 59.

[0429] That is to say, when the electronic device 900 wakes up the virtual assistant on the first page 87, and displays the first status window 60 corresponding to the virtual assistant, and at least one first control 59 adjacent to the first status window 60, the user can perform an input operation on the first status window 60, or can perform a clicking operation on the first control 59 to interact with the virtual assistant.

[0430] Further, based on other wake-up methods, after the electronic device 900 wakes up the virtual assistant on the first page 87, the electronic device 900 can also have other operation forms.

[0431] The fourth wake-up method is used to wake up the virtual assistant by the user, and the embodiment of the application is further described.

[0432] FIG. 24 is a schematic diagram of a display method of a second control provided by the embodiment of the application.

[0433] As shown in FIG. 18, FIG. 19 and FIG. 24, an example is illustrated in the way that the electronic device 900 employs gesture navigation.

[0434] The user can wake up the virtual assistant on the first page 87 in the way of dragging the target page content 90 of the first page 87 to the navigation bar 61.

[0435] In the case that the first moving area 92 reaches the second transition window 96, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first state window 60 corresponding to the virtual assistant. The scenario that the user further drags the first moving area 92 to the first state window 60 is shown in FIG. 19.

[0436] In fact, the user can not need to use the target page content 90 in the first moving area 92 in real time. Then, in the embodiment of the present application, the user can not further drag the first moving area 92 to the first state window 60, but drag the first moving area 92 to the adjacent area of the first state window 60, so that, in response to the dragging of the first moving area 92 to the adjacent area of the first state window 60, the electronic device 900 generates a second control 115 in the adjacent area of the first state window 60, and the second control 115 is used to cache the target page content 90 for subsequent use. Wherein, the second control 115 can be arranged side by side with the first control 59.

[0437] It should be noted here that the first control 59 can not only provide service recommendations for the content in the first page 87, but also provide service recommendations for the content in the second control 115.

[0438] It should be further noted that the embodiment shows the way of generating the second control 115 based on the dragging operation in the case that the virtual assistant is not woken up. In fact, the embodiment of the present application can also generate the second control 115 based on the dragging operation in the case that the virtual assistant is woken up. The two ways can be referred to each other, and the present application will not be described here. In the case that the virtual assistant is woken up, the way of generating the second control 115 based on the dragging operation realizes the collaborative use of the virtual assistant and the first page 87.

[0439] The user can perform a click operation on the second control 115 when the target page content 90 is needed, and in response to the click operation of the user on the second control 115, the electronic device 900 can take the second control 115 as the input of the second state window 62.

[0440] FIG. 25 is a schematic diagram of the updating process of the first control according to an embodiment of the present application.

[0441] As shown in FIG. 25, in an implementation manner, the electronic device 900 can update the first control 59 based on the target page content 90 in the second control 115, to obtain an updated control 116, and the updated control 116 includes a service recommendation matched with the target page content 90.

[0442] For example, the first control 59 initially provides the service recommendations of "polish the article" and "summarize the abstract" for the first page 87. If the target page content 90 is text content in a non-native language, the first control 59 is updated to the updated control 116, and the service recommendation provided by the updated control 116 for the second control 115 is, for example, "translation".

[0443] That is, the electronic device 900 can wake up the virtual assistant on the first page 87, display the first state window 60 corresponding to the virtual assistant, and display at least one updated control 116 adjacent to the first state window 60, and display the second control 115 adjacent to the first control 59.

[0444] In this way, the user can perform an input operation on the first state window 60, a click operation on the updated control 116, and a click operation on the second control 115 to interact with the virtual assistant.

[0445] The embodiments of the present application are further described below with the user waking up the virtual assistant in the first manner.

[0446] In the first manner, the user can perform an input operation on the first state window 60 or a click operation on the first control 59 to interact with the virtual assistant.

[0447] FIG. 26 is a schematic diagram of a scenario in which the user performs an input operation on the first state window according to an embodiment of the present application.

[0448] As shown in FIG. 26, in the case of displaying the first state window 60, the user can perform a click operation on the input field 99. In response to the click operation on the input field 99, the electronic device 900 displays the input method panel 117, and the input method panel 117 covers the first state window 60 so that the first state window 60 is not visible.

[0449] After the user completes the input operation, the user can perform an exit operation on the input method panel 117. The exit operation can be, for example, a gesture operation or a click operation on the exit button 118. In response to the operation of exiting the input method panel 117, the electronic device 900 hides the input method panel 117, displays the second state window 62, and displays the interaction content corresponding to the input operation in the second state window 62.

[0450] It should be noted that the input method panel 117 is actually a system-level pop-up control, and similar system-level pop-up controls also include a slidable panel control and the like, and the display priority of such controls can be higher than that of the first state window 60. After exiting such controls, the electronic device 900 can display the first state window 60 again. The first window 58 in the remaining states can have the same display priority as the first state window 60. Details are not described herein.

[0451] It should be noted that the first state window 60 has an automatic exit mechanism (which will be described in subsequent embodiments). During the process in which the electronic device 900 displays the input method panel 117, the automatic exit mechanism of the first state window 60 is suspended. The first window 58 in the remaining states can be the same. Details are not described herein.

[0452] In the above embodiments, after the user completes the input operation, the electronic device 900 switches from displaying the first state window 60 to displaying the second state window 62. In the embodiments, there are various switching modes between the first window 58 in different states in different scenarios. The switching mode of the first window 58 is described as follows.

[0453] Specifically, during the process in which the user interacts with the virtual assistant, the electronic device 900 can switch the state of the first window 58 based on automatic switching or based on a gesture operation, so as to meet different use requirements of the user.

[0454] FIG. 27 is a first scenario in which the first window switches states, according to an embodiment of the present application.

[0455] As shown in FIG. 27, the user can perform an input operation on the first state window 60. In response to the input operation on the first state window 60, the electronic device 900 displays the first sub-window 104 and displays the interaction content corresponding to the input operation in the first panel 106.

[0456] It should be noted that the input operation can be an operation based on the input method panel 117 or an operation based on the voice input button 102. The specific operation form of the input operation is not limited in the embodiments.

[0457] Based on different instructions corresponding to the input operation, the electronic device 900 can provide a response content, automatically perform a corresponding operation corresponding to the response content, or jump to other application programs based on the response content, so as to enable the user to use other application programs quickly.

[0458] Taking the input operation based on the voice input button 102 as an example, the user inputs a voice instruction of "turn on Bluetooth" based on the voice input button 102 to the first state window 60, and the electronic device 900 displays the first sub-window 104 and displays the text content converted from the voice instruction in the first sub-window 104. Based on the virtual assistant, the electronic device 900 can display a response content in the first sub-window 104, and the response content can be, for example, "Bluetooth has been turned on for you". In addition, the electronic device 900 can automatically perform a corresponding operation corresponding to the response content.

[0459] In order to ensure that the automatically performed operation meets the actual needs of the user, the electronic device 900 can display a response control associated with the response content in the first sub-window 104, and the response control can be, for example, the Bluetooth control 119, which can include the Bluetooth switch 120 displayed in an on state to indicate that the electronic device 900 has currently automatically performed the operation of turning on Bluetooth. If the operation automatically performed by the electronic device 900 does not meet the needs of the user, the user can manually turn off the response control.

[0460] FIG. 28 is a schematic diagram of a second scenario of the first window performing window state switching according to an embodiment of the present application.

[0461] As shown in FIG. 28, taking the input operation based on the input method panel 117 as an example, the user inputs text content of "introduce artificial intelligence" based on the input method panel 117, and the electronic device 900 displays the first sub-window 104 and displays the response content corresponding to the text content in the first sub-window 104.

[0462] Further, if the user inputs a voice instruction of "store the current content to the note application" based on the voice input button 102 to the first sub-window 104, the electronic device 900 can jump to the note application and display the response content corresponding to the text content on a page in the note application. In the embodiment of the present application, the electronic device 900 supports jumping to any application by the virtual assistant, and the type of the application that can be jumped to is not limited in the embodiment of the present application.

[0463] FIG. 29 is a schematic diagram of a third scenario of the first window performing window state switching according to an embodiment of the present application.

[0464] As shown in FIG. 29, if the window size of the first sub-window 104 cannot accommodate the response content, the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105 and displays the response content in the second panel 108.

[0465] Specifically, the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105, which can be triggered when the electronic device 900 detects that the display area of the current interaction content is greater than the preset first area threshold. The first area threshold may, for example, be equal to 80% of the total window area of the first sub-window. The present application does not limit the specific value of the first area threshold.

[0466] That is, the electronic device 900 can automatically trigger the switching from displaying the first sub-window 104 to displaying the second sub-window 105 based on the interaction content, so that the second sub-window 105 provides a larger display area for the interaction content, improving the user's visual experience.

[0467] It should be noted here that after the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105, the subsequent interaction content can be displayed in the second sub-window 105, and the electronic device 900 no longer automatically triggers the switching from displaying the second sub-window 105 to displaying the first sub-window 104.

[0468] The electronic device 900 can also switch the window state of the first window 58 based on the manner of the gesture operation.

[0469] FIG. 30 is a fourth scenario diagram of the window state switching of the first window according to an embodiment of the present application.

[0470] As shown in FIG. 30, when the first sub-window 104 is displayed, if the user thinks that the display area of the first sub-window 104 is too small, the user can perform an upward sliding operation on the navigation bar 61 in the first sub-window 104. In response to the upward sliding operation on the navigation bar 61 in the first sub-window 104, the electronic device 900 can display a third state window 97.

[0471] The interaction content in the first sub-window 104 can be synchronously displayed in the third state window 97, so that the third state window 97 provides a larger display area for the interaction content, improving the user's visual experience.

[0472] FIG. 31 is a fifth scenario diagram of the window state switching of the first window according to an embodiment of the present application.

[0473] As shown in FIG. 31, the electronic device 900 can automatically trigger the switching from displaying the first sub-window 104 to displaying the second sub-window 105 based on the interaction content.

[0474] When the user considers that the display area of the second sub-window 105 is too small, the user can perform an upward sliding operation on the navigation bar 61 in the second sub-window 105. In response to the upward sliding operation on the navigation bar 61 in the second sub-window 105, the electronic device 900 can display a third state window 97.

[0475] The interactive content in the second sub-window 105 can be synchronously displayed in the third state window 97. In this way, the third state window 97 provides a larger display area for the interactive content, and improves the visual experience of the user.

[0476] It should be noted that the embodiments of the present application only exemplarily describe the manner in which the first sub-window 104 is switched to the third state window 97, and the manner in which the second sub-window 105 is switched to the third state window 97. In fact, in the embodiments of the present application, when the first state window 60 is displayed, in response to the upward sliding operation on the navigation bar 61 in the first state window 60, the electronic device 900 can display at least one of the first sub-window 104, the second sub-window 105, and the third state window 97. When the first sub-window 104 is displayed, in response to the upward sliding operation on the navigation bar 61 in the first sub-window 104, the electronic device 900 can display at least one of the second sub-window 105 and the third state window 97.

[0477] The window displayed by the electronic device 900 can be determined based on the position at which the user performs the upward sliding operation. The electronic device 900 can set the switching manner between the first window 58 in any two display states to support gesture switching, or can set the switching manner between the first window 58 in any two display states to not support gesture switching, which is not limited in the embodiments of the present application.

[0478] FIG. 32 is a sixth scene diagram of window state switching of the first window according to an embodiment of the present application.

[0479] As shown in FIG. 32, when the electronic device 900 displays the third state window 97, if the user wants to simultaneously operate the first page 87, the user can switch the third state window 97 to the first state window 60.

[0480] The user can perform a downward sliding operation on the top area of the display screen of the electronic device 900. In response to the downward sliding operation on the top area of the display screen of the electronic device 900, the electronic device 900 switches from displaying the third state window 97 to displaying the first state window 60.

[0481] It should be noted that the embodiments of the present application only exemplarily describe the manner in which the third state window 97 switches to the first state window 60. In fact, in the embodiments of the present application, when the third state window 97 is displayed, in response to the downward swipe operation on the top area of the display screen of the electronic device 900, the electronic device 900 can display at least one of the first state window 60, the first sub-window 104, and the second sub-window 105. When the second sub-window 105 is displayed, in response to the downward swipe operation on the navigation bar 61 in the second sub-window 105, the electronic device 900 can display at least one of the first state window 60 and the first sub-window 104. When the first sub-window 104 is displayed, in response to the downward swipe operation on the navigation bar 61 in the first sub-window 104, the electronic device 900 can display the first state window 60.

[0482] The window displayed by the electronic device 900 can be determined based on the position at which the user performs the downward swipe operation. The electronic device 900 can set the switching manner between the first windows 58 of any two display states to support gesture switching, or can set the switching manner to not support gesture switching, which is not limited in the embodiments of the present application. For example, if the electronic device 900 sets the switching between the third state window 97 and the second sub-window 105 to not support the downward swipe operation, the electronic device 900 cannot switch from the third state window 97 to the second sub-window 105 in response to the downward swipe operation.

[0483] FIG. 33 is a schematic diagram of a seventh scenario in which the first window switches the window state according to an embodiment of the present application.

[0484] As shown in FIG. 33, when the electronic device 900 displays the third state window 97, if the user wants to end the virtual assistant session, the user can perform a downward swipe operation on the navigation bar 61 in the third state window 97. In response to the downward swipe operation on the navigation bar 61 in the third state window 97, the electronic device 900 closes the third state window 97 and displays the navigation bar 61 at the bottom of the display screen.

[0485] Correspondingly, when the electronic device 900 displays the second state window 62, if the user wants to end the virtual assistant session, the user can perform a downward swipe operation on the navigation bar 61 in the second state window 62. In response to the downward swipe operation on the navigation bar 61 in the second state window 62, the electronic device 900 closes the second state window 62 and displays the navigation bar 61 at the bottom of the display screen. The specific implementation manner can be referred to the third state window 97, and the present application does not show the diagram.

[0486] It should be noted that in the second state window 62 and the third state window 97, if there is interaction content, the second state window 62 and the third state window 97 can cache the interaction content to generate history content, which is displayed when the virtual assistant is subsequently awakened again.

[0487] Correspondingly, in the case that the electronic device 900 displays the first state window 60, if the user wants to end the virtual assistant session, the user can perform a downward sliding operation on the navigation bar 61 in the first state window 60. In response to the downward sliding operation on the navigation bar 61 in the first state window 60, the electronic device 900 closes the first state window 60 and displays the navigation bar 61 at the bottom of the display screen. This specific implementation mode can be referred to the third state window 97, and the present application does not show the figure.

[0488] Further, in the process of the user interacting with the virtual assistant, the electronic device 900 can also move the first window 58 based on the gesture operation mode to meet different use requirements of the user.

[0489] FIG. 34 is a first scene diagram of the electronic device moving the first window provided by the embodiment of the present application.

[0490] As shown in FIG. 34, in the case of displaying the first state window 60, the user can perform a long press and drag operation on the navigation bar 61 in the first state window 60. In response to the long press and drag operation on the navigation bar 61 in the first state window 60, the electronic device 900 displays the first state window 60 as a corresponding moving state window 121. The moving state window 121 includes the first interaction area 98 and a fourth panel 122, and the fourth panel 122 is used to accommodate the first control 59.

[0491] The user can drag the moving state window 121 to a target position 123. In response to the operation of dragging the moving state window 121 to the target position 123, the electronic device 900 displays the moving state window 121 at the target position 123.

[0492] The display mode of the moving state window 121 is more suitable for display in the central region of the display screen, which can improve the visual experience and operation experience of the user. When the user performs a long press and drag operation on the navigation bar 61 in the first state window 60, the navigation bar 61 can change in dynamic effect, for example, change in color, such as from gray to blue. The specific mode in which the navigation bar 61 can change in dynamic effect is not limited by the embodiment of the present application.

[0493] FIG. 35 is a second scene diagram of the electronic device moving the first window provided by the embodiment of the present application.

[0494] As shown in FIG. 35, in the case of displaying the first sub-window 104, the user can perform a long-pressing and dragging operation on the navigation bar 61 in the first sub-window 104, and in response to the operation of long-pressing and dragging the navigation bar 61 in the first sub-window 104 to the target position 123, the electronic device 900 displays the first sub-window 104 at the target position 123.

[0495] In the operation of long-pressing and dragging the navigation bar 61 in the first sub-window 104 by the user, the navigation bar 61 can change in dynamic effect, for example, change in color, such as from gray to blue.

[0496] The first sub-window 104 has a proper window size, so that no morphological change is needed for the first sub-window 104 in the process of moving the first sub-window 104.

[0497] FIG. 36 is a schematic diagram of a fourth scenario of moving a first window of an electronic device according to an embodiment of the present application.

[0498] As shown in FIG. 36, in the case of displaying the second sub-window 105, the user can perform a long-pressing and dragging operation on the navigation bar 61 in the second sub-window 105, and in response to the operation of long-pressing and dragging the navigation bar 61 in the second sub-window 105, the electronic device 900 reduces the height of the second panel 108 to the height of the first panel 106, so that the second sub-window 105 is transformed into the first sub-window 104. The user can drag the first sub-window 104 to the target position 123, and in response to the operation of dragging the first sub-window 104 to the target position 123, the electronic device 900 displays the first sub-window 104 at the target position 123.

[0499] In the operation of long-pressing and dragging the navigation bar 61 in the second sub-window 105 by the user, the navigation bar 61 can change in dynamic effect, for example, change in color, such as from gray to blue.

[0500] The window size of the second sub-window 105 occupies a large proportion in the display screen, so that morphological change is needed for the second sub-window 105 in the process of moving the second sub-window 105, which can improve the visual experience of the user.

[0501] It should be noted that the first scenario to the third scenario of moving a first window of an electronic device described above have the same implementation manner for foldable electronic devices and non-foldable electronic devices, and the embodiments of the present application are exemplarily described by taking non-foldable electronic devices. The implementation manner of foldable electronic devices can be referred to the non-foldable electronic devices.

[0502] FIG. 37 is a schematic diagram of a fourth scenario of moving a first window of an electronic device according to an embodiment of the present application.

[0503] As shown in Figure 37, when the third state window 97 is displayed, the user can long-press and drag the top area of ​​the display screen. In response to this long-press and drag operation, the electronic device 900 can display the third state window 97 as the second state window 62. The user can drag the second state window 62 to the target position 123. In response to dragging the second state window 62 to the target position 123, the electronic device 900 displays the second state window 62 at the target position 123.

[0504] In the non-foldable electronic device, the electronic device 900 can display the third state window 97 as the first sub-window 104. In the foldable electronic device, the electronic device 900 can display the third state window 97 as the second sub-window 105. This display method can be determined based on the difference in screen area between the non-foldable and foldable electronic devices. In fact, in the foldable electronic device, the electronic device 900 can also display the third state window 97 as the first sub-window 104. The specific display form of the second state window 62 is not limited in this embodiment. This embodiment is only illustrated by example using a foldable electronic device.

[0505] Figure 38 is a schematic diagram of the fifth scenario of the electronic device moving the first window according to an embodiment of this application.

[0506] As shown in Figure 38, when the electronic device 900 is a foldable electronic device and is in an unfolded state, and the third state window 97 is displayed in the full-screen area corresponding to all displays, the user can long press and drag along the top edge of the display. In response to the operation of long pressing and dragging along the top edge of the display, based on the split-screen mode, the electronic device 900 can display the third state window 97 in the full-screen area corresponding to a part of the display, that is, in the split screen on one side, and display the first page 87 in the full-screen area corresponding to another part of the display, that is, in the split screen on the other side.

[0507] The above describes the user's input operations on the first state window 60 under the first wake-up method, as well as the window state switching operations and window movement operations performed after the input operations.

[0508] Furthermore, in the first wake-up method, the user can also click on the first control 59. In response to the user's click on the first control 59, the electronic device 900 can execute the service recommendation function in the first control 59.

[0509] Figure 39 is a schematic diagram of the first scenario provided by the embodiment of this application, showing a user interacting with a virtual assistant based on a first control.

[0510] As shown in FIG. 39, in the case that the service recommendation in the first control 59 can provide a text service (for example, a summary abstract), the user performs a click operation on the first control 59, and in response to the click operation on the first control 59, the electronic device 900 displays the first sub-window 104 and displays the text service content corresponding to the service recommendation in the first sub-window 104.

[0511] If the window size of the first sub-window 104 cannot accommodate the text service content, the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105, and displays the text service content in the second panel 108. The manner in which the first sub-window 104 switches to display the second sub-window 105 in this embodiment can refer to the foregoing embodiments, and will not be described herein. The embodiments of this application are only exemplarily described by displaying the first sub-window 104.

[0512] It should be noted that the service recommendation in the first control 59 is recommended according to the current content of the first page 87, and if the current content of the first page 87 is updated, the first control 59 is updated accordingly.

[0513] For example, when the first page 87 is a mail page, if the current content of the first page 87 is the body of the mail, the first control 59 can provide a service recommendation according to the body of the mail, and the service recommendation can be, for example, "polish the article" or "summary abstract". If the current content of the first page 87 is the mail address, the first control 59 can provide a service recommendation according to the mail address, and the service recommendation can be, for example, "navigation" or "make a call".

[0514] If there is no selected area in the first page 87, the first control 59 can provide a service recommendation according to the overall content currently displayed in the first page 87. If there is a selected area in the first page 87, for example, the first selected area 91 in the foregoing embodiments, the first control 59 can provide a service recommendation according to the selected area in the first page 87.

[0515] It should also be noted that the embodiments of this application not only support providing a service recommendation for a selected area in the first page 87 through the first control 59 when the virtual assistant is woken up, but also can provide a service recommendation for a selected area in the first page 87 through the first control 59 when the virtual assistant is not woken up. In the case that the virtual assistant is not woken up, the electronic device 900 only displays the navigation bar 61 at the bottom of the first page 87, and does not display the first window 58 in any window form, and then the electronic device 900 can display the first control 59 in the adjacent area of the navigation bar 61.

[0516] Further, in order to be closer to the use intention of the user, the electronic device 900 can display the first control 59 in the adjacent area of the navigation bar 61 in the case that the selected area exists in the first page 87 and the user further performs long pressing on the selected area to call out the copy button. The specific triggering form of displaying the first control 59 is not limited in the embodiments of the present application.

[0517] In addition, in order to give a strong prompt to the user, the first control 59 can display a dynamic effect, for example, a flowing light dynamic effect. The dynamic effect type of the first control 59 is not limited in the embodiments of the present application.

[0518] In fact, in the case that the virtual assistant is not woken up, the electronic device 900 displays the first control 59 in the adjacent area of the navigation bar 61 may not meet the user's needs, then, if the user does not operate the first control 59 within a preset time length, the electronic device 900 hides the first control 59. The preset time length may be 5s, for example. The user can also perform a downward sliding operation on the first control 59, and in response to the downward sliding operation, the electronic device 900 hides the first control 59.

[0519] FIG. 40 is a second scenario diagram of the user interacting with the virtual assistant based on the first control according to an embodiment of the present application.

[0520] As shown in A of FIG. 40, in the case that the service recommendation in the first control 59 can provide a third-party service (for example, shopping), the user performs a click operation on the first control 59, and in response to the click operation on the first control 59, the electronic device 900 jumps to a shopping application, displays a shopping page 124a corresponding to the shopping application, and ends the virtual assistant session.

[0521] As shown in B of FIG. 40, based on a plurality of first controls 59, the electronic device 100 can provide a plurality of third-party services, which can be the same type of service or different types of services. The type of the provided third-party service is not limited in the embodiments of the present application.

[0522] Taking the case of providing the same type of service by the electronic device 100 as an example, in the case of providing a shopping service, each first control 59 can be used to jump to a different shopping application. In the first control 59, a shopping T icon that can jump to a shopping application T1, a shopping D icon that can jump to a shopping application D1, and a shopping P icon that can jump to a shopping application P1 can be displayed.

[0523] The user can perform a click operation on any first control 59, for example, on the first control 59 corresponding to the shopping T icon. In response to the click operation on the first control 59, the electronic device 900 jumps to the shopping application T1 and displays the shopping page 124b corresponding to the shopping application T1. In addition, the first state window 60 can also be displayed on the shopping page 124b, and the remaining first controls 59 except the clicked first control 59, for example, the first control 59 corresponding to the shopping D icon and the first control 59 corresponding to the shopping P icon, can be displayed in the adjacent area of the first state window 60. In this way, for the same type of service, the electronic device 900 provides multiple selection methods for the user, which can meet the user's needs.

[0524] Further, within a certain preset time length, if the user does not click the remaining first controls 59, the electronic device 900 hides the remaining first controls 59, for example, the first control 59 corresponding to the shopping D icon and the first control 59 corresponding to the shopping P icon.

[0525] FIG. 41 is a third scenario diagram of user interaction with a virtual assistant based on a first control according to an embodiment of the present application.

[0526] As shown in FIGS. 34 and 41, if the user performs a moving operation on the first state window 60 in the first wake-up mode, the electronic device 900 currently displays the moving state window 121. If the first control 59 in the moving state window 121 provides a third-party service (for example, shopping), when the user performs a click operation on the first control 59, in response to the click operation on the first control 59, the electronic device 900 jumps to the shopping application and displays the shopping page 124 corresponding to the shopping application, and displays the moving state window 121 in the shopping page 124. That is, unlike the embodiment corresponding to FIG. 40, if the first control 59 corresponding to the moving state window 121 jumps to the shopping application, the electronic device can continue to provide the virtual assistant service based on the moving state window 121, and the virtual assistant session will not end. The first control 59 can further provide service recommendations to the shopping page 124 to meet the user's use needs.

[0527] The above describes the way in which the user interacts with the virtual assistant on the first page 87 in the first wake-up mode.

[0528] Further, after interacting with the virtual assistant, the user can further interact with the first page 87 to realize the collaborative use of the virtual assistant and the first page 87.

[0529] As shown in FIGS. 27-38, after the user interacts with the virtual assistant, the first page 87 can currently display any one of the first status window 60, the second status window 62, or the third status window 97.

[0530] It should be noted that the user interaction with the virtual assistant not only refers to the user waking up the virtual assistant, but also includes the user inputting, moving, switching, and the like on the first window 58 corresponding to the virtual assistant.

[0531] If the user only interacts with the virtual assistant based on the wake-up operation, the first page 87 can display the first status window 60 based on the user's long-press operation on the navigation bar 61, or display the second status window 62 based on the user's drag operation. The first page 87 usually does not directly display the third status window 97, and the third status window 97 is usually displayed based on the user's sliding gesture operation.

[0532] During the user's further interaction with the first page 87, the display state of the first status window 60, the second status window 62, and the third status window 97 (which can refer to the third status window 97 in the split-screen mode) does not change, regardless of whether the first status window 60, the second status window 62, or the third status window 97 is currently displayed. That is, the electronic device 900 does not end the session of the virtual assistant.

[0533] The following embodiments of the present application exemplarily illustrate the case where, after the user interacts with the virtual assistant, the electronic device 900 displays the first status window 60 on the first page 87. In this case, the user can perform various forms of operations on the first page 87.

[0534] For example, in the case where the first application is an email application, the first page 87 usually displays text content, and the user can perform up-swing and down-swing operations on the first page 87 to browse the text content. The user can also perform operations of returning to a higher-level page and entering a lower-level page on the first page 87 to jump between different hierarchical pages in the email application. The user can also perform operations of jumping to other application pages and returning to the first page 87, and operations of jumping to other system-level pages (such as the notification center and the multi-task page) and returning to the first page 87. The user can also perform selection operations and drag operations on the first page to copy or save the text content. The embodiments of the present application do not limit the form of the user's operation on the first page.

[0535] FIG. 42 is a schematic diagram of a first scenario in which the user performs an operation on the first page according to an embodiment of the present application.

[0536] As shown in FIG. 42, in a case where the electronic device 900 displays the first state window 60, the user can perform a downward sliding operation on the first page 87, and in response to the user's downward sliding operation on the first page 87, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain a first updated page 125.

[0537] Further, based on the first updated page 125, the electronic device 900 can update the first control 59 to obtain an updated control 126. The updated control 126 includes a service recommendation matched with the first updated page 125.

[0538] Specifically, the electronic device 900 usually needs a certain update time to update the first control 59, that is, the electronic device 900 can update the first control 59 based on the page content in the first updated page 125 within a first preset time length after the user stops the downward sliding operation, and display the updated control 126 when the first preset time length is reached. For example, the first preset time length can be 3 seconds, and the specific value of the first preset time length is not limited in the embodiments of the present application.

[0539] For example, the electronic device displays Chinese content 127 in the first page 87, and displays the first state window 60 and the first control 59. The service recommendation provided by the first control 59 based on the Chinese content 127 can be "polish the article" and "summarize the abstract", for example.

[0540] The user performs a downward sliding operation on the first page 87, and in response to the user's downward sliding operation on the first page 87, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain the first updated page 125. The first updated page 125 can display English content 128, for example.

[0541] After 3 seconds, the electronic device 900 can update the first control 59 based on the English content 128 in the first updated page 125 to obtain the updated control 126. The service recommendation provided by the updated control 126 based on the English content 128 can be "translation", for example.

[0542] In this way, the electronic device 900 can update the corresponding service recommendation in real time according to the updated content in the first page 87. Moreover, the hiding mechanism of hiding the first control 59 can not only improve the user's visual experience, but also ensure the adaptability of the provided service recommendation.

[0543] FIG. 43 is a second scenario diagram of the operation on the first page provided by the embodiments of the present application.

[0544] As shown in FIG. 43, in a case where the electronic device 900 displays the first state window 60, the user can perform a selection operation on the target page content 90 in the first page 87, and in response to the user's selection operation on the target page content 90, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain a first updated page 125.

[0545] Further, based on the first updated page 125, the electronic device 900 can update the first control 59 to obtain an updated control 126. The updated control 126 includes a service recommendation matched with the first updated page 125.

[0546] Specifically, the electronic device 900 usually needs a certain update time to update the first control 59, that is, the electronic device 900 can update the first control 59 based on the target page content 90 within a first preset time length after the user stops the selection operation, and display the updated control 126 when the first preset time length is reached, where the first preset time length may, for example, be 3 seconds, and the specific value of the first preset time length is not limited in the embodiments of the present application.

[0547] For example, the electronic device displays Chinese content 127 and English content 128 in the first page 87, displays the first state window 60, and displays the first control 59, where the service recommendation provided by the first control 59 based on the Chinese content 127 may, for example, be "polish the article" and "summarize the abstract".

[0548] The user can perform a selection operation on the English content 128, and in response to the user's selection operation on the English content 128, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain the first updated page 125. The first updated page 125 may, for example, be a page highlighting the English content 128.

[0549] After 3 seconds, the electronic device 900 can update the first control 59 based on the English content 128 in the first updated page 125 to obtain the updated control 126, where the service recommendation provided by the updated control 126 based on the English content 128 may, for example, be "translation".

[0550] In this way, the electronic device 900 can update the corresponding service recommendation in real time according to the updated content in the first page 87. Moreover, the above-mentioned hiding mechanism of the first control 59 can not only improve the user's visual experience, but also ensure the adaptability of the provided service recommendation.

[0551] Further, in embodiments of the present application, the electronic device 900 can set the priority of different operation forms, for example, the priority of the selected operation can be set to be greater than the priority of the sliding operation, so that in the case that the user has selected the content of a region in the first page 87, if the user performs the sliding operation again, the electronic device 900 can preferentially update the service recommendation in real time based on the selected operation. The setting manner of the priority of different operation forms in embodiments of the present application is not limited.

[0552] FIG. 44 is a schematic diagram of a third scenario of user operation on the first page according to an embodiment of the present application.

[0553] As shown in FIGS. 43 and 44, after the user performs the selected operation on the first page 87 and the electronic device 900 updates the first control 59 to obtain the updated control 126, if the user further performs the long-pressing and copying operation on the selected region, the electronic device 900 updates the updated control 126 based on the copying operation to obtain the updated control 129. The updated control 129 can provide the service recommendation related to the copying operation.

[0554] Further, as shown in the foregoing embodiments, the service recommendation provided by the updated control 126 based on the selected region can be “translation”. Then, the electronic device 900 updates the updated control 126 based on the copying operation to obtain the updated control 129, wherein the service recommendation provided by the updated control 129 based on the copying operation can be “save notes”.

[0555] That is, the priority of the copying operation can be set to be greater than the priority of the selected operation in the embodiment.

[0556] It should be noted that the manner in which the electronic device 900 updates the service recommendation in real time under other operation forms (for example, the downward sliding operation, the operation of returning to the upper page, and the operation of entering the lower page) can refer to the foregoing embodiments, and the present application will not be described herein.

[0557] Further, as known from the embodiments corresponding to FIGS. 42 and 43, the electronic device 900 sets the hiding mechanism for the first control 87, and in embodiments of the present application, the electronic device 900 also sets the hiding mechanism for the first window 58 to improve the visual experience of the user.

[0558] FIG. 45 is a schematic diagram of a fourth scenario of user operation on the first page according to an embodiment of the present application.

[0559] As shown in FIG. 45, in a case where the electronic device 900 displays the first status window 60, the user can perform an operation of swiping from a bottom edge to a top edge of the display screen, and in response to the user operation of swiping from the bottom edge to the top edge of the display screen, the electronic device 900 displays the multitasking page 130 and hides the first status window 60 and the first control 59. Further, the user can perform an operation of returning from the multitasking page 130 to the first page 87 again, and in response to the user operation of returning from the multitasking page 130 to the first page 87 again, the electronic device 900 displays the first page 87.

[0560] It should be noted here that the hiding mechanism provided by the electronic device 900 in other operation forms (for example, jumping to other system-level pages such as a notification control center page, a minus one screen page, a right two screen page, etc., and the type of the system-level page is not limited by the embodiments of the present application) can be referred to the embodiments, and the present application will not be described here.

[0561] In this way, based on the above operation mode, the user can use the virtual assistant and the first application program in coordination.

[0562] Since the user can interact in the first page 87 and the first status window 60 multiple times, in order to avoid the conflict of gesture operations in the interaction process, the embodiments of the present application set the related mechanism of focus switching to ensure the accuracy of identifying the user behavior.

[0563] FIG. 46 is a first scene diagram of focus switching provided by the embodiments of the present application.

[0564] As shown in FIG. 46, a first scene of the user operating the first page is exemplarily illustrated.

[0565] In a case where the user performs a downward swipe operation on the first page 87, in response to the user downward swipe operation on the first page 87, the electronic device 900 can set a first focus on the first page 87 to focus on the first page 87, and then control the downward swipe operation to match the first page 87. That is, the current downward swipe operation will not affect the first status window 60.

[0566] In a case where the downward swipe operation matches the first page 87, the electronic device 900 controls the first page 87 to update corresponding to the downward swipe operation to obtain a first updated page 125.

[0567] At this time, the electronic device 900 displays the first state window 60 as a defocus state. In the defocus state, the first state window 60 does not respond to the current gesture operation, and the first window dynamic effect 103 corresponding to the defocus state is different from the normal state. For example, the first window dynamic effect 103 can be a dynamic effect of morphological reduction and no color frame display. The embodiments of the present application do not limit the specific type of the first window dynamic effect 103 corresponding to the defocus state.

[0568] Correspondingly, in the case where the second state window 62 is displayed on the first page 87, the window dynamic effect corresponding to the defocus state is different from the normal state. For example, the window dynamic effect can be a dynamic effect of no color frame display. The embodiments of the present application only exemplarily illustrate the defocus state for the first state window 60. FIG. 47 is a schematic diagram of a second scenario of focus switching provided by the embodiments of the present application.

[0569] As shown in FIGS. 46 and 47, the user can perform a click operation on the first state window 60 again. In response to the click operation on the first state window 60, the electronic device 900 moves the first focus to the first state window 60, so that the first state window 60 is displayed from the defocus state to the focus state.

[0570] In the case where the first state window 60 is displayed as the focus state, the electronic device 900 focuses on the first state window 60, and then controls at least one of the up sliding operation, the down sliding operation, and the input operation to match the first state window 60. In the case where the related operation with respect to the first state window 60 matches the first state window 60, the electronic device 900 controls the first state window 60 to respond to the related operation.

[0571] In the focus state, the first window dynamic effect 103 of the first state window 60 can restore the initial dynamic effect.

[0572] Further, in the case where the electronic device 900 is a foldable electronic device and the foldable electronic device is in an unfolded state, since the first page 87 can display more content, the related mechanism of focus switching can be further set for the first page 87 in the split-screen mode.

[0573] FIG. 48 is a schematic diagram of a third scenario of focus switching provided by the embodiments of the present application.

[0574] As shown in A of FIG. 48, in the case where the first page 87 is displayed based on the split-screen mode and the first state window 60 is currently displayed, if the first focus stays in the first split-screen page 131 corresponding to the first page 87, the electronic device 900 updates the first control 59 based on the first split-screen page 131 to obtain an updated control 132, and the updated control 132 includes service recommendations matched with the first split-screen page 131.

[0575] For example, the first split-screen page 131 includes Chinese content 127, and the service recommendation provided by the updated control 132 based on the Chinese content 127 can be, for example, "polish the article" or "summarize the abstract".

[0576] As shown in B of FIG. 48, if the first focus is on the second split-screen page 133 corresponding to the first page 87, the electronic device 900 updates the first control 59 based on the second split-screen page 133 to obtain the updated control 132, and the updated control 132 includes a service recommendation matched with the second split-screen page 133.

[0577] For example, the first split-screen page 131 includes English content 128, and the service recommendation provided by the updated control 132 based on the English content 128 can be, for example, "translation".

[0578] It should be noted that after the user wakes up the virtual assistant, the user can only perform a click operation on the first page 87, and the electronic device 900 sets the first focus on the first page 87 and provides a service recommendation corresponding to the split-screen page where the first focus is located. In fact, the above process is also applicable to the first updated page 125 in the foregoing embodiments. The embodiments of the present application only exemplarily describe the first page 87, and the two pages can be referred to each other, and the embodiments of the present application will not be described here.

[0579] FIG. 49 is a fourth scene diagram of focus switching provided by an embodiment of the present application.

[0580] As shown in FIGS. 48 and 49, on the basis of the scene shown in FIG. 48, the first page 87 can further display a floating window 134. If the first focus is on the floating window 134, the electronic device 900 updates the first control 59 based on the floating window 134 to obtain the updated control 132, and the updated control 132 includes a service recommendation matched with the floating window 134.

[0581] After the user performs the above interaction process with the virtual assistant, the user can end the session with the virtual assistant.

[0582] FIG. 50 is a first scene diagram of ending a virtual assistant session provided by an embodiment of the present application.

[0583] As shown in FIG. 50, a first scene in which the user performs an operation on the first page is exemplarily described.

[0584] In the case that the electronic device 900 displays the first state window 60, the user can perform a downward sliding operation on the first page 87. During the user's interaction with the first page 87, if the user does not use the virtual assistant within a certain time, it indicates that the user currently can not have the intention to use the virtual assistant. Therefore, the electronic device 900 can minimize the first state window 60 to the navigation bar 61 to end the session of the virtual assistant. In addition, since the virtual assistant is still displayed on the first page 87 when the user performs the operation on the first page 87, the user can not need to use the virtual assistant again after the interaction with the virtual assistant. Therefore, the first state window 60 stays on the first page 87 for a long time, which can easily affect the visual experience of the user.

[0585] To this end, the embodiments of the present application provide an automatic exit mechanism for the virtual assistant.

[0586] Specifically, in the case that the preset time length is reached after the virtual assistant is woken up, the electronic device 900 can minimize the first state window 60 to the navigation bar 61. The preset time length can be, for example, 10 seconds. The embodiments of the present application do not limit the specific value of the preset time length.

[0587] If there is interaction content between the user and the virtual assistant before the virtual assistant exits, the first state window 60 can cache the interaction content to generate historical content. The first window 58 in other states is the same, and the embodiments of the present application do not repeat the description. The way in which the virtual assistant exits based on the gesture operation has been described in the foregoing embodiments, and the embodiments of the present application do not repeat the description.

[0588] FIG. 51 is a second scenario diagram for ending the session of the virtual assistant according to an embodiment of the present application.

[0589] As shown in FIG. 51, the user can also exit the virtual assistant based on other operation forms. For example, the user can perform an upward sliding operation from the bottom of the display screen of the electronic device 900. In response to the upward sliding operation from the bottom of the display screen of the electronic device 900, the electronic device 900 returns to the home screen page 57 from the first application and ends the session of the virtual assistant.

[0590] If the user needs to use the virtual assistant again for the interaction process, the user can perform the wake-up operation on the virtual assistant again.

[0591] FIG. 52 is a scenario diagram for waking up the virtual assistant again according to an embodiment of the present application.

[0592] As shown in A of FIG. 52, in the first wake-up mode, the user can wake up the virtual assistant again based on the navigation bar 61. If the user has interacted with the virtual assistant before and the virtual assistant has cached the interaction content, in response to the user's operation of waking up the virtual assistant again, the electronic device 900 displays the first sub-window 104 (or the second sub-window 105) corresponding to the virtual assistant, and displays the historical content in the first sub-window 104.

[0593] As shown in B of FIG. 52, the user can wake up the virtual assistant again based on the navigation bar 61. Whether the user has interacted with the virtual assistant before or not, the electronic device 900 displays the first state window 60. That is, the electronic device 900 does not need to display the historical content. If the electronic device 900 is set in this form, the electronic device 900 does not need to cache the interaction content when the user ends the virtual assistant session in the foregoing embodiments.

[0594] The method shown in the embodiments of the present application further supports the operation of the page in the first application again after the virtual assistant is woken up in the first application, that is, the first application and the virtual assistant can operate cooperatively, and the virtual assistant can provide the service recommendation matched with the updated page. The present application improves the user experience in the use of the virtual assistant.

[0595] The embodiments of the present application further provide a virtual assistant cooperative use device, applied to an electronic device.

[0596] The virtual assistant cooperative use device comprises:

[0597] A display module, configured to display a first page of a first application.

[0598] A wake-up module, configured to wake up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control comprising a service recommendation matched with the first page.

[0599] A page updating module, configured to update the first page to obtain a first updated page in the first application in response to a first operation on the first page.

[0600] A control updating module, configured to update the first control based on the first updated page to obtain a first updated control, the first updated control comprising a service recommendation matched with the first updated page.

[0601] The virtual assistant cooperative use device provided in the embodiments of the present application supports reoperating the page in the first application program after the virtual assistant is woken up in the first application program, that is, the first application program and the virtual assistant can cooperatively operate, and the virtual assistant can provide service recommendations matched with the updated page, thereby improving the user experience in the process of using the virtual assistant.

[0602] The embodiments of the present application further provide an electronic device, including a processor and a memory. The memory stores program instructions. When the program instructions are executed by the processor, the electronic device performs the virtual assistant cooperative use method in any implementation manner in the above embodiments.

[0603] FIG. 53 is a structural schematic diagram of a chip system provided in the embodiments of the present application.

[0604] As shown in FIG. 53, the chip system 300 provided in the embodiments of the present application, for example, a SoC, includes at least one processor 301 and at least one interface circuit 302. The processor 301 and the interface circuit 302 can be interconnected through a line. For example, the interface circuit 302 can be used to receive signals from other devices (for example, the memory of the electronic device). For another example, the interface circuit 302 can be used to send signals to other devices (for example, the processor 301 or the touch screen of the electronic device). Illustratively, the interface circuit 302 can read the instructions stored in the memory and send the instructions to the processor 301. When the instructions are executed by the processor 301, the electronic device can perform various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0605] The embodiments of the present application further provide a computer readable storage medium, which includes computer instructions. When the computer instructions run on the above electronic device, the electronic device performs various functions or steps in the above method embodiments.

[0606] The embodiments of the present application further provide a computer program product. When the computer program product runs on the computer, the computer performs various functions or steps in the above method embodiments.

[0607] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions.

[0608] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the division of the apparatus embodiments is merely an example, and for example, the division of the modules or units can be different, and for example, multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0609] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, i.e., may be located in one place, or may be distributed in multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0610] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0611] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, and includes several instructions to make a device (which can be a single chip, a chip, etc.) or a processor (Processor) execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0612] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A virtual assistant cooperative use method characterized by, The application is applied to an electronic device, comprising: displaying a first page of a first application; waking up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control comprising a service recommendation matching the first page; updating the first page in response to a first operation on the first page to obtain a first updated page in the first application; updating the first control based on the first updated page to obtain a first updated control, the first updated control comprising the service recommendation matching the first updated page. 2.The virtual assistant collaborative use method of claim 1, wherein, The waking up of the virtual assistant on the first page comprises: waking up the virtual assistant on the first page in response to a second operation on a navigation button, wherein the navigation button comprises at least one of a navigation bar, a home button in a navigation bar, and the second operation comprises at least one of a long press operation and a click operation; or, waking up the virtual assistant on the first page in response to a third operation on a power key, the third operation comprising at least one of a long press operation and a click operation; or, waking up the virtual assistant on the first page in response to an operation of dragging target page content of the first page to the navigation button. 3.The virtual assistant collaborative use method of claim 2, wherein, The waking up of the virtual assistant on the first page in response to the operation of dragging the target page content of the first page to the navigation button comprises: generating a first selected area at a position where the target page content is located in response to a selected operation on the target page content; generating a first moving area for copying and moving the target page content in response to a long press operation on the first selected area; displaying a transition window at a first position corresponding to the navigation button in response to an operation of dragging the first moving area to the navigation button, the transition window being used to prompt a user that the virtual assistant can be woken up at the first position; waking up the virtual assistant on the first page in a case where the first moving area reaches a first preset area, wherein the first preset area is an area adjacent to the transition window, and the first preset area is used to avoid a screen hot area. 4.The virtual assistant collaborative use method of claim 3, wherein, The waking up of the virtual assistant on the first page in the case where the first moving area reaches the first preset area further comprises: generating a second control in a region adjacent to the first window of the first window in response to an operation of dragging the first moving area to the region adjacent to the first window, the second control being used to cache the target page content. The generation of the second control in the region adjacent to the first window of the first window further comprises: 5.The virtual assistant collaborative use method of claim 4, wherein, updating the first control based on the target page content in the second control to obtain a second updated control, the second updated control comprising the service recommendation matching the target page content. The display of the first window corresponding to the virtual assistant comprises: 6.The virtual assistant collaborative use method of claim 2, wherein, ​ In response to the second operation on the navigation button, or in response to the operation of dragging the page content of the first page to the navigation button, the first window is displayed in a first input state, and the first input state is a state in which no voice input is currently received. In response to the third operation on the power key, the first window is displayed in a second input state, and the second input state is a state in which voice input is currently received. 7.The virtual assistant collaborative use method of claim 2, wherein, The display of the first window corresponding to the virtual assistant includes: In the case where the first page displays the navigation bar, the first window is displayed in an adjacent area of the navigation bar to avoid the navigation bar.

8. The virtual assistant collaborative use method according to any one of claims 2-7, characterized in that, Further comprising: In the case where the first window is displayed, a first interaction area is displayed in the first window, and the first interaction area is used to receive an input operation of a user. In response to a click operation on the first interaction area, an input method panel is displayed, and the input method panel covers the first window so that the first window is not visible. In response to an operation of exiting the input method panel, the input method panel is hidden, and the first window is displayed.

9. The virtual assistant cooperative use method of any one of claims 2-7, wherein: The first window includes a first state window, a second state window, and a third state window. The first state window includes a first interaction area, and the first interaction area is used to receive an input operation of a user. The second state window includes a first sub-window and a second sub-window, the first sub-window includes the first interaction area and a first panel, the first panel is displayed in a part of a display screen of the electronic device, the first panel is used to accommodate interaction content between the virtual assistant and the user, the second sub-window includes the first interaction area and a second panel, the second panel is displayed in a part of the display screen of the electronic device, and a panel height of the second panel is higher than a panel height of the first panel, the second panel is used to accommodate the interaction content between the virtual assistant and the user. The third state window includes the first interaction area and a third panel, the third panel is displayed in a full-screen area corresponding to at least one of the display screens, and the third panel is used to accommodate the interaction content between the virtual assistant and the user. The display of the first window corresponding to the virtual assistant and at least one first control adjacent to the first window includes: 10.The virtual assistant collaborative use method of claim 9, wherein, The first state window is displayed, and the first control is displayed in an outer peripheral adjacent area of the first state window. Alternatively, the first sub-window is displayed, and the first control is displayed in the first panel, and the first control is displayed in an outer peripheral adjacent area of the first interaction area. After the display of the first window corresponding to the virtual assistant and at least one first control adjacent to the first window, further comprising: 11.The virtual assistant collaborative use method of claim 10, wherein, ​ In response to an input operation on the first status window, display the first sub-window and display interaction content corresponding to the input operation in the first panel; In the first panel, if an area of a display region corresponding to the interaction content is greater than a first area threshold, display the second sub-window and display the interaction content in the second panel. 12.The virtual assistant collaborative use method of claim 10, wherein, The display of the first status window further comprises: In response to a fourth operation on the first control, display the first sub-window and display interaction content corresponding to the first control in the first panel, the fourth operation comprising at least one of a click operation and a drag operation; In the first panel, if an area of a display region corresponding to the interaction content is greater than a first area threshold, display the second sub-window and display the interaction content in the second panel. 13.The virtual assistant collaborative use method according to claim 11 or 12, characterized in that, Further comprising: In a case where the first status window is displayed, in response to an upward swipe operation on a navigation bar in the first status window, display at least one of the first sub-window, the second sub-window, and the third status window; In a case where the first sub-window is displayed, in response to an upward swipe operation on a navigation bar in the first sub-window, display at least one of the second sub-window and the third status window; In a case where the second sub-window is displayed, in response to an upward swipe operation on a navigation bar in the second sub-window, display the third status window. 14.The virtual assistant collaborative use method according to claim 11 or 12, characterized in that, Further comprising: In a case where the third status window is displayed, in response to a downward swipe operation on a top region of the display screen, display at least one of the first status window, the first sub-window, and the second sub-window, or minimize the third status window to the navigation button; In a case where the second sub-window is displayed, in response to a downward swipe operation on a navigation bar in the second sub-window, display at least one of the first status window and the first sub-window, or minimize the second sub-window to the navigation button; In a case where the first sub-window is displayed, in response to a downward swipe operation on a navigation bar in the first sub-window, display the first status window, or minimize the first sub-window to the navigation button. 15.The virtual assistant collaborative use method of claim 10, wherein, The display of the first status window further comprises: In response to a long press and drag operation on the navigation bar in the first status window, display the first status window as a corresponding mobile status window, the mobile status window comprising the first interaction region and a fourth panel for accommodating the first control; In response to an operation of dragging the mobile status window to a target position, display the mobile status window at the target position. 16.The virtual assistant collaborative use method of claim 11 or 12, wherein, The display of the second sub-window further comprises: In response to a long press and drag operation on the navigation bar in the second sub-window, reduce a panel height of the second panel to a panel height of the first panel, so that the second sub-window is transformed into the first sub-window; In response to the operation of dragging the first sub-window to a target position, the first sub-window is displayed at the target position. 17.The virtual assistant collaborative use method of claim 13, wherein, After the third state window is displayed, the method further includes: In response to the operation of long pressing and dragging a top region of the display screen, the third state window is displayed as the second state window. In response to the operation of dragging the second state window to a target position, the second state window is displayed at the target position. 18.The virtual assistant collaborative use method of claim 13, wherein, After the third state window is displayed, the method further includes: In a case where the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the third state window is displayed in a full-screen region corresponding to the entire display screen, in response to the operation of long pressing and dragging along a top edge of the top region of the display screen, the third state window is displayed in a full-screen region corresponding to a part of the display screen based on a split-screen mode, and the first page is displayed in a full-screen region corresponding to another part of the display screen. The method further includes: 19.The virtual assistant collaborative use method of claim 11 or 12, wherein, In a case where the first page is updated in response to the first operation on the first page, if the first state window or the second state window is displayed on the first page, the display states of the first state window and the second state window remain unchanged. The updating of the first page in response to the first operation on the first page to obtain a first updated page in the first application includes:

20. The virtual assistant collaborative use method of any one of claims 1-19, wherein, In response to the first operation on the first page, the first page is updated, and the first control is hidden. In response to the stopping of the first operation, the first updated page is obtained. The first operation includes at least one of an up-swipe operation, a down-swipe operation, an operation of returning to a higher-level page, an operation of entering a lower-level page, and an operation of jumping to a second page and then returning to the first page, the second page being a page other than a page in the first application. The updating of the first control based on the first updated page to obtain a first updated control includes:

21. The virtual assistant collaborative use method of claim 20, wherein, Within a first preset time period after the stopping of the first operation, the first control is updated based on the first updated page. In a case where the first preset time period is reached, the first updated control is displayed. The updating of the first page in response to the first operation on the first page to obtain a first updated page in the first application includes:

22. The virtual assistant collaborative use method of any one of claims 1-19, wherein, In response to the first operation on the first page, a first focus is set on the first page to focus on the first page, so as to control the first operation to match the first page. In a case where the first operation matches the first page, the first page is controlled to be updated according to the first operation to obtain the first updated page. After the first focus is set on the first page in response to the first operation on the first page to focus on the first page, the method further includes:

23. The virtual assistant collaborative use method of claim 22, wherein, The first window is displayed in a defocused state. ​ In response to a click operation on the first window, the first focus is moved to the first window so that the first window is displayed from the defocused state to a focused state; In a case where the first window is displayed in the focused state, the first window is focused, and then the first window is controlled The fifth operation is matched with the first window; In a case where the fifth operation is matched with the first window, the first window is controlled to respond to the fifth operation, the fifth operation including at least one of an up-swipe operation, a down-swipe operation, and an input operation. 24.The virtual assistant collaborative use method of claim 23, wherein, The first control is updated based on the first updated page to obtain a first updated control, including: In a case where the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the first updated page is displayed based on a split-screen mode, if the first focus stays in a first split-screen page corresponding to the first updated page, the first control is updated based on the first split-screen page to obtain the first updated control, the first updated control including the service recommendation matched with the first split-screen page; if the first focus stays in a second split-screen page corresponding to the first updated page, the first control is updated based on the second split-screen page to obtain the first updated control, the first updated control including the service recommendation matched with the second split-screen page. 25.The virtual assistant collaborative use method of claim 23, wherein, The first control is updated based on the first updated page to obtain a first updated control, including: In a case where the first updated page includes a floating window, and the first focus stays in the floating window, the first control is updated based on the floating window to obtain the first updated control, the first updated control including the service recommendation matched with the floating window.

26. The virtual assistant collaborative use method of claim 11 or 12, wherein, After the first control is updated based on the first updated page to obtain a first updated control, the method further includes: If the second preset time period is reached after the virtual assistant is woken up, the first window is minimized to the navigation button, and the interaction content is cached as historical content.

27. The virtual assistant collaborative use method of claim 26, wherein, After the first window is minimized to the navigation button, and the interaction content is cached as historical content, the method further includes: The virtual assistant is woken up again on the first page to display a first window corresponding to the virtual assistant, and the historical content is displayed in the first window.

28. The virtual assistant collaborative use method of any one of claims 1-27, wherein, After the first control is updated based on the first updated page to obtain a first updated control, the method further includes: In response to an up-swipe operation performed by a bottom of a display screen of the electronic device, the first application is returned to a home screen page, and a session of the virtual assistant is ended.

29. An electronic device, comprising: The method includes: A processor and a memory, the memory storing program instructions that, when executed by the processor, cause the electronic device to perform the method for cooperative use of a virtual assistant as claimed in any one of claims 1-28.

30. A computer-readable storage medium, comprising: The computer instructions, when executed on an electronic device, cause the electronic device to perform the method for cooperative use of a virtual assistant as claimed in any one of claims 1-28. ​

Images