Virtual assistant display method and electronic device

Abstract

The present application provides a virtual assistant display method and an electronic device. The method comprises: displaying a first page; waking up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control comprising a service recommendation matching the first page; in response to a first operation performed on the first page, updating the first page to obtain a first updated page; and, on the basis of the first updated page, updating the first control to obtain a first updated control, the first updated control comprising a service recommendation matching the first updated page. In this way, when a user does not input any instruction, a virtual assistant can provide a service recommendation to the user, to enable the user to quickly use services on the basis of the service recommendation. When the user further performs a page update operation, the virtual assistant can update the service recommendation to enable the user to quickly use services on the basis of the updated service recommendation. The technical solution shown in the present application improves user experience.

Description

Virtual assistant display method and electronic device

[0001] This application claims priority to International Patent Application No. PCT / CN2024 / 119220, filed on September 14, 2024, entitled "Virtual assistant collaborative use method and electronic device", and to Chinese Patent Application No. 202411493376.0, filed on October 23, 2024, entitled "Virtual assistant display method", the contents of both of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminals, and in particular to a virtual assistant display method and an electronic device. BACKGROUND

[0003] A virtual assistant is an interactive service implemented through artificial intelligence technology. A user usually needs to input instructions to the virtual assistant through voice input or keyboard input, so that the virtual assistant provides services based on the instructions input by the user.

[0004] Currently, the virtual assistant cannot provide service recommendations to the user in the case where the user does not input instructions, so that the user cannot quickly use services based on service recommendations, the interaction between the user and the virtual assistant is not convenient, and the user experience is affected. SUMMARY

[0005] The present application provides a virtual assistant display method and an electronic device. In the case where the user does not input instructions, the virtual assistant can provide service recommendations to the user, so that the user can quickly use services based on service recommendations. When the user further performs an operation (for example, an operation of updating a page, or an operation of inputting multi-modal content to the virtual assistant), the virtual assistant can update the service recommendations, so that the user can quickly use services based on the updated service recommendations. The technical solution disclosed in the present application improves the user experience.

[0006] In a first aspect, the present application provides a virtual assistant display method applied to an electronic device, including: displaying a first page; waking up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control including service recommendations matched with the first page; in response to a first operation on the first page, updating the first page to obtain a first updated page; updating the first control based on the first updated page to obtain a first updated control, the first updated control including service recommendations matched with the first updated page.

[0007] The method shown in the embodiments of the present application can provide service recommendations to the user by the virtual assistant without inputting instructions by the user, so that the user can quickly use the service based on the service recommendations, and the virtual assistant can update the service recommendations when the user further updates the page, so that the user can quickly use the service based on the updated service recommendations. The technical solution shown in the present application improves the user experience.

[0008] In an implementation manner, the virtual assistant is woken up on the first page, including: in response to a second operation on a navigation button, the virtual assistant is woken up on the first page, wherein the navigation button includes at least one of a navigation bar, a home button in a navigation bar, and the second operation includes at least one of a long press operation and a click operation; or, in response to a third operation on a power key, the virtual assistant is woken up on the first page, and the third operation includes at least one of a long press operation and a click operation; or, in response to an operation of dragging the first target content of the first page to the navigation button, the virtual assistant is woken up on the first page; or, in response to an operation of dragging the first target content to the first screen area, a first icon corresponding to the virtual assistant is displayed on the first screen area, and in response to an operation of dragging the first target content to the first icon, the virtual assistant is woken up on the first page. With the implementation manner, multiple ways of waking up the virtual assistant are shown to meet different use habits of users and improve the user experience in using the virtual assistant.

[0009] In an implementation manner, the virtual assistant is woken up on the first page in response to an operation of dragging the first target content of the first page to the navigation button, including: in response to a selection operation on the first target content, a first selected area is generated at a position where the first target content is located; in response to a long press operation on the first selected area, a first moving area is generated, and the first moving area is used to copy and move the first target content; in response to an operation of dragging the first moving area to the navigation button, a transition window is displayed at a first position corresponding to the navigation button, and the transition window is used to prompt the user that the virtual assistant can be woken up at the first position; and in a case where the first moving area reaches a first preset area, the virtual assistant is woken up on the first page, wherein the first preset area is an area adjacent to the transition window, and the first preset area is used to avoid a screen hot area. With the implementation manner, the virtual assistant can be woken up based on the dragging manner, and the screen hot area can be avoided in the process, thereby improving the user experience in using the virtual assistant.

[0010] In an implementation, in a case where the first moving area reaches the first preset area, the virtual assistant is woken up on the first page, and then the method further includes: in response to the first moving area being dragged to an adjacent area of the first window, generating a second control in the adjacent area of the first window, the second control being used for caching the first target content. With this implementation, the first target content dragged by the user can be cached for subsequent use by the user, which conforms to the operation habit of the user and improves the use experience of the user in using the virtual assistant.

[0011] In an implementation, after the second control is generated in the adjacent area of the first window, the method further includes: updating the first control based on the first target content in the second control to obtain a second updated control, the second updated control including a service recommendation matched with the first target content. With this implementation, the service recommendation can be refreshed based on the first target content dragged by the user, so that the service recommendation conforms to the demand of the user and improves the use experience of the user in using the virtual assistant.

[0012] In an implementation, displaying the first window corresponding to the virtual assistant includes: in a case where the virtual assistant is woken up in response to a second operation on a navigation button or in response to the first target content of the first page being dragged to the navigation button, displaying the first window with a first input state, the first input state being a state in which no voice input is currently received; in a case where the virtual assistant is woken up in response to a third operation on a power key, displaying the first window with a second input state, the second input state being a state in which voice input is currently received. With this implementation, different input states of the first window are provided based on different wake-up manners of the user, which adapts to the use habit of the user and improves the use experience of the user in using the virtual assistant.

[0013] In an implementation, displaying the first window corresponding to the virtual assistant includes: in a case where a navigation bar is displayed on the first page, displaying the first window in an adjacent area of the navigation bar to avoid the navigation bar. With this implementation, the first window and the navigation bar are avoided, which adapts to the use habit of the user and improves the use experience of the user in using the virtual assistant.

[0014] In an implementation, the method further includes: in a case where the first window is displayed, displaying a first interaction area in the first window, the first interaction area being configured to receive an input operation of the user; in response to a click operation on the first interaction area, displaying the input method panel, the input method panel covering the first window so that the first window is not visible; and in response to an operation of exiting the input method panel, hiding the input method panel and displaying the first window. With this implementation, the first window and the input method panel are not displayed at the same time, which improves the aesthetics of the page, and this setting mode adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0015] In an implementation, the first window includes a first state window, a second state window, and a third state window; the first state window includes a first interaction area configured to receive an input operation of the user; the second state window includes a first sub-window and a second sub-window, the first sub-window includes a first interaction area and a first panel, the first panel being displayed in a part of the display screen of the electronic device, the first panel being configured to accommodate interaction content between the virtual assistant and the user, the second sub-window includes a first interaction area and a second panel, the second panel being displayed in a part of the display screen of the electronic device, and a panel height of the second panel being higher than a panel height of the first panel, the second panel being configured to accommodate interaction content between the virtual assistant and the user; and the third state window includes a first interaction area and a third panel, the third panel being displayed in a full-screen area corresponding to at least one display screen, the third panel being configured to accommodate interaction content between the virtual assistant and the user. With this implementation, the first window is provided in multiple states, and this setting mode adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0016] In an implementation, displaying the first window corresponding to the virtual assistant and at least one first control adjacent to the first window includes: displaying the first state window and displaying the first control in an outer-adjacent area of the first state window; or displaying the first sub-window and displaying the first control in the first panel, and the first control being displayed in an outer-adjacent area of the first interaction area. With this implementation, the initial display mode of the first window is shown, and this setting mode adapts to the user's usage habits and improves the user's experience in using the virtual assistant.

[0017] In an implementation, after displaying the first window corresponding to the virtual assistant and the at least one first control adjacent to the first window, the method further includes: in response to an input operation on the first state window, displaying a first sub-window and displaying, in the first panel, interactive content corresponding to the input operation; and in the first panel, if an area of a display region corresponding to the interactive content is greater than a first area threshold, displaying a second sub-window and displaying, in a second panel, the interactive content. With this implementation, the switching manner between the first windows in different states is shown, and this setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0018] In an implementation, after displaying the first window corresponding to the virtual assistant and the at least one first control adjacent to the first window, the method further includes: in response to a fourth operation on the first control, displaying a first sub-window and displaying, in the first panel, interactive content corresponding to the first control, the fourth operation including at least one of a click operation and a drag operation; and in the first panel, if an area of a display region corresponding to the interactive content is greater than a first area threshold, displaying a second sub-window and displaying, in a second panel, the interactive content. With this implementation, the switching manner between the first windows in different states is shown, and this setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0019] In an implementation, the method further includes: in a case where the first state window is displayed, in response to an upward sliding operation on a navigation bar in the first state window, displaying at least one of the first sub-window, the second sub-window, and a third state window; in a case where the first sub-window is displayed, in response to an upward sliding operation on a navigation bar in the first sub-window, displaying at least one of the second sub-window and the third state window; and in a case where the second sub-window is displayed, in response to an upward sliding operation on a navigation bar in the second sub-window, displaying the third state window. With this implementation, the switching manner between the first windows in different states is shown, and this setting manner adapts to the use habit of the user, thereby improving the use experience of the user in using the virtual assistant.

[0020] In an implementation, after displaying the third state window, the method further includes: in response to a downward swipe operation on a top region of the display screen, displaying at least one of the first state window, the first sub-window, and the second sub-window, or minimizing the third state window to the navigation button; in response to a downward swipe operation on a navigation bar in the second sub-window, displaying at least one of the first state window and the first sub-window, or minimizing the second sub-window to the navigation button; in response to a downward swipe operation on a navigation bar in the first sub-window, displaying the first state window, or minimizing the first sub-window to the navigation button. With this implementation, the manner of switching between the first windows in different states is shown, and this setting manner adapts to the use habits of users, thereby improving the use experience of users in using the virtual assistant.

[0021] In an implementation, after displaying the first state window, the method further includes: in response to a long press and drag operation on a navigation bar in the first state window, displaying the first state window as a corresponding moving state window of the first state window, the moving state window including the first interaction region and a fourth panel, the fourth panel being configured to accommodate the first control; and in response to a drag operation of the moving state window to a target position, displaying the moving state window at the target position. With this implementation, the manner of moving the first state window and the manner of changing the first state window during the moving process are shown, and this setting manner adapts to the use habits of users, thereby improving the use experience of users in using the virtual assistant.

[0022] In an implementation, after displaying the second sub-window, the method further includes: in response to a long press and drag operation on a navigation bar in the second sub-window, reducing a panel height of the second panel to a panel height of the first panel, so that the second sub-window is transformed into the first sub-window; and in response to a drag operation of the first sub-window to a target position, displaying the first sub-window at the target position. With this implementation, the manner of moving the second sub-window and the manner of changing the second sub-window during the moving process are shown, and this setting manner adapts to the use habits of users, thereby improving the use experience of users in using the virtual assistant.

[0023] In an implementation, after displaying the third state window, the method further includes: in response to a long press and drag operation on a top region of the display screen, displaying the third state window as the second state window; and in response to a drag operation of the second state window to a target position, displaying the second state window at the target position. With this implementation, the manner of moving the third state window and the manner of changing the third state window during the moving process are shown, and this setting manner adapts to the use habits of users, thereby improving the use experience of users in using the virtual assistant.

[0024] In an implementation manner, after the third state window is displayed, the method further includes: in a case that the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the third state window is displayed in a full-screen area corresponding to the entire display screen, in response to a long-pressing and dragging operation along the top edge of the top area of the display screen, displaying the third state window in a full-screen area corresponding to a part of the display screen and displaying the first page in a full-screen area corresponding to another part of the display screen based on the split-screen mode. By adopting the implementation manner, the manner of moving the third state window in the foldable electronic device is shown, and the change manner of the third state window in the moving process is shown. The setting manner adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0025] In an implementation manner, the method further includes: in a case that the first page is updated in response to the first operation on the first page, if the first state window or the second state window is displayed on the first page, the display state of the first state window and the second state window remains unchanged. By adopting the implementation manner, the display state of the first state window and the second state window remains unchanged in the process of operating the first page, and therefore, the first application program and the virtual assistant can be cooperatively operated.

[0026] In an implementation manner, the first page is updated in response to the first operation on the first page to obtain a first updated page, including: the first page is updated in response to the first operation on the first page, and the first control is hidden; the first updated page is obtained in response to stopping the first operation; and the first operation includes at least one of an up-swipe operation, a down-swipe operation, an operation of returning to a higher-level page, an operation of entering a lower-level page, and an operation of jumping to a second page and then returning to the first page. By adopting the implementation manner, a hiding mechanism is provided in the process of updating the first page, and the visual experience of the user is avoided from being affected by the first control that is not adapted. The setting manner adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0027] In an implementation manner, the first control is updated based on the first updated page to obtain a first updated control, including: the first control is updated based on the first updated page within a first preset time length after the first operation is stopped; and the first updated control is displayed in a case that the first preset time length is reached. By adopting the implementation manner, the service recommendation based on the first updated page can be matched based on the preset time length, and it is ensured that the service recommendation in the first updated control is more adapted. The setting manner adapts to the use habit of the user, and the use experience of the user in the process of using the virtual assistant is improved.

[0028] In an implementation, in response to the first operation on the first page, the first page is updated to obtain a first updated page, including: in response to the first operation on the first page, setting a first focus on the first page to focus on the first page, so as to control the first operation to match the first page; in a case where the first operation matches the first page, controlling the first page to be updated according to the first operation to obtain the first updated page. With this implementation, based on the setting of the focus, it can be ensured that the user operates in the required area, and this setting mode adapts to the user's usage habits, thereby improving the user experience in using the virtual assistant.

[0029] In an implementation, after setting the first focus on the first page to focus on the first page in response to the first operation on the first page, the method further includes: displaying the first window in a defocused state; in response to a click operation on the first window, moving the first focus to the first window to display the first window from the defocused state to a focused state; in a case where the first window is displayed in the focused state, focusing on the first window, so as to control the fifth operation to match the first window; in a case where the fifth operation matches the first window, controlling the first window to respond to the fifth operation, the fifth operation including at least one of an up sliding operation, a down sliding operation, and an input operation. With this implementation, based on the moving of the focus, it can be ensured that the user operates in the required area, and this setting mode adapts to the user's usage habits, thereby improving the user experience in using the virtual assistant.

[0030] In an implementation, the first control is updated based on the first updated page to obtain a first updated control, including: in a case where the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the first updated page is displayed based on a split-screen mode, if the first focus stays in a first split-screen page corresponding to the first updated page, updating the first control based on the first split-screen page to obtain the first updated control, the first updated control including service recommendations matching the first split-screen page; if the first focus stays in a second split-screen page corresponding to the first updated page, updating the first control based on the second split-screen page to obtain the first updated control, the first updated control including service recommendations matching the second split-screen page. With this implementation, based on the moving of the focus, it can be ensured that the user operates in the required area, and this setting mode adapts to the user's usage habits, thereby improving the user experience in using the virtual assistant.

[0031] In an implementation, the first control is updated based on the first updated page to obtain a first updated control, including: in a case where the first focus is on a floating window included in the first updated page, the first control is updated based on the floating window to obtain the first updated control, the first updated control including a service recommendation matched with the floating window. With this implementation, the mobile focus-based manner can ensure that the user operates in a desired area, and this setting manner adapts to the user's usage habits, improving the user's experience in using the virtual assistant.

[0032] In an implementation, after the first control is updated based on the first updated page to obtain the first updated control, the method further includes: if a second preset time length is reached after the virtual assistant is woken up, minimizing the first window to a navigation button and caching the interaction content as historical content. With this implementation, the session of the virtual assistant can be automatically ended, and this setting manner adapts to the user's usage habits, improving the user's experience in using the virtual assistant.

[0033] In an implementation, after the first window is minimized to the navigation button and the interaction content is cached as the historical content, the method further includes: the virtual assistant is woken up again on the first page to display the first window corresponding to the virtual assistant, and the historical content is displayed in the first window. With this implementation, the historical content can be retained in the virtual assistant, meeting the user's usage needs and conforming to the user's operation habits.

[0034] In an implementation, after the first control is updated based on the first updated page to obtain the first updated control, the method further includes: in response to an operation of swiping up from the bottom of the display screen of the electronic device, ending the session of the virtual assistant. With this implementation, the home screen page can be returned based on the user's operation, and this setting manner adapts to the user's usage habits, improving the user's experience in using the virtual assistant.

[0035] In an implementation, after the virtual assistant is woken up on the first page to display the first window corresponding to the virtual assistant, the method further includes: in response to sending a second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third updated control associated with the second target content in the first window, where each third updated control is used to provide at least one service recommendation matched with the second target content. With this implementation, the virtual assistant can provide service recommendations to the user without the user inputting instructions, so that the user can quickly use the service based on the service recommendations. When the user further performs an operation of inputting multi-modal content to the virtual assistant, the virtual assistant can update the service recommendations, so that the user can quickly use the service based on the updated service recommendations. The technical solution shown in this application improves the user's experience.

[0036] In an implementation, in response to a box selection operation on the target content, a first box selection area is generated at a position corresponding to the target content, so that the target content in the first box selection area can perform a drag operation, wherein the target content includes a first target content and a second target content. With this implementation, the target content that cannot be dragged can be changed to target content that can be dragged based on a user operation, so that the virtual assistant can provide service recommendations for the target content that can be dragged, thereby improving the user experience.

[0037] In an implementation, in a case where the first page is a home screen page and the first status window is displayed, a first attachment area is displayed in a region adjacent to the first status window, wherein the first attachment area includes at least one attachment entry, and the attachment entry is used to provide second target content. With this implementation, the virtual assistant has different display modes for different pages to adapt to different scenarios. In this implementation, since the home screen page usually does not have second target content, the virtual assistant provides a first attachment area to facilitate the user to send second target content to the virtual assistant based on the entry of the area, thereby meeting the different operation habits of the user and improving the user experience.

[0038] In an implementation, the response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window includes: in response to a click operation on a target attachment entry in the attachment entry, displaying an attachment page corresponding to the target attachment entry and displaying the second target content in the attachment page; in response to a selection operation on the second target content, sending the second target content to the virtual assistant to display the second status window on the first page and display the second target content in the second status window, and display the third update control in the second status window. With this implementation, the user can send the second target content to the virtual assistant based on the attachment entry. This application provides multiple ways to send the second target content to the virtual assistant to meet the different operation habits of the user and improve the user experience.

[0039] In an implementation, the displaying the second target content in the first window and the at least one third update control associated with the second target content in the first window in response to sending the second target content on the first page to the virtual assistant includes, in response to an operation of dragging the second target content to the first state window, changing the display of the first state window to the display of the second state window, displaying the second target content in the second state window, and displaying the third update control in the second state window. With this implementation, after the virtual assistant is woken up, the user can send the second target content to the virtual assistant based on the manner of dragging the second target content, so as to meet different operation habits of the user and improve the user experience.

[0040] In an implementation, the displaying the second target content in the first window and the at least one third update control associated with the second target content in the first window in response to sending the second target content on the first page to the virtual assistant further includes, in a case where the second target content is a picture, in response to a click operation on a first target control in the third update control, displaying a first target service recommendation corresponding to the first target control in the first window, and uploading the first target service recommendation to a first server, the first server including a visual question and answer (VQA) server; in a process of uploading the first target service recommendation to the first server, displaying a first response dialog box corresponding to the first target service recommendation in the first window, and displaying first waiting information in the first response dialog box, the first waiting information being used to prompt that the electronic device is in a process of obtaining a first VQA result corresponding to the first target service recommendation; and after obtaining the first VQA result issued by the first server for the first target service recommendation, displaying the first VQA result in the first response dialog box. With this implementation, in the process of uploading the picture, the uploading process does not need to be displayed in the foreground of the electronic device, but is integrated into the first waiting information, so as to improve the visual experience of the user.

[0041] In an implementation, in the process of uploading the first target service recommendation to the first server, the first response dialog box corresponding to the first target service recommendation is displayed in the first window, and after the first waiting information is displayed in the first response dialog box, the method further includes: displaying first error information in the first response dialog box when the first VQA result issued by the first server for the first target service recommendation is not acquired, displaying a first error button in a region adjacent to the first target service recommendation, and the first error information is used to prompt the user that the first server is abnormal; in response to a click operation on the first error button, the first error button is stopped from being displayed, and the first waiting information is displayed in the first response dialog box; and after the first VQA result issued by the first server for the first target service recommendation is acquired, the first VQA result is displayed in the first response dialog box. With the implementation, the electronic device adds a text instruction retransmission mechanism, so that the user can resend the text instruction based on the first error button, the operation is simple, and the user experience is improved.

[0042] In an implementation, in the process of uploading the first target service recommendation to the first server, the first response dialog box corresponding to the first target service recommendation is displayed in the first window, and after the first waiting information is displayed in the first response dialog box, the method further includes: displaying first error information in the first response dialog box when the first VQA result issued by the first server for the first target service recommendation is not acquired, displaying a first error button in a region adjacent to the first target service recommendation, and the first error information is used to prompt the user that the first server is abnormal; in response to a click operation on the first error button, the first error button is stopped from being displayed, and the first waiting information is displayed in the first response dialog box; and after the first VQA result issued by the first server for the first target service recommendation is acquired, the first VQA result is displayed in the first response dialog box. With the implementation, the electronic device adds a text instruction retransmission mechanism, so that the user can resend the text instruction based on the first error button, the operation is simple, and the user experience is improved.

[0043] In an implementation, in response to sending the second target content on the first page to the virtual assistant, the second target content is displayed in the first window, and after the at least one third update control associated with the second target content is displayed in the first window, the method further includes: in response to a click operation on the second target control in the third update control, displaying a second target service recommendation corresponding to the second target control in the first window, and uploading the second target service recommendation to a second server, the second server including a document question and answer (DQA) server; during the uploading of the second target service recommendation to the second server, displaying a third response dialog box corresponding to the second target service recommendation in the first window, and displaying third waiting information in the third response dialog box, the third waiting information being used to prompt that the electronic device is in a process of obtaining a first DQA result corresponding to the second target service recommendation; and after obtaining the first DQA result issued by the second server for the second target service recommendation, displaying the first DQA result in the third response dialog box. With this implementation, during the uploading of the document, the uploading process does not need to be displayed in the foreground of the electronic device, but is integrated into the third waiting information, thereby improving the visual experience of the user.

[0044] In an implementation, during the uploading of the second target service recommendation to the second server, after the third response dialog box corresponding to the second target service recommendation is displayed in the first window and the third waiting information is displayed in the third response dialog box, the method further includes: when the first DQA result issued by the second server for the second target service recommendation is not obtained, displaying second error information in the third response dialog box, and displaying a second error button in a region adjacent to the second target service recommendation, the second error information being used to prompt that the second server is abnormal; in response to a click operation on the second error button, stopping displaying the second error button, and displaying the third waiting information in the third response dialog box; and after obtaining the first DQA result issued by the second server for the second target service recommendation, displaying the first DQA result in the third response dialog box. With this implementation, the electronic device adds a text instruction retransmission mechanism, so that the user can retransmit the text instruction based on the second error button, the operation is simple, and the user experience is improved.

[0045] In an implementation manner, in the process of uploading the second target service recommendation to the second server, the third response dialog box corresponding to the second target service recommendation is displayed in the first window, and after the third waiting information is displayed in the third response dialog box, the second error information is further displayed in the third response dialog box when the first DQA result issued by the second server for the second target service recommendation is not acquired, and the second error button is displayed in the region adjacent to the second target service recommendation, the second error information is used to prompt the user that the second server is abnormal; in response to the seventh operation of inputting the second instruction into the first window, the display of the second error button is stopped, the fourth response dialog box corresponding to the second instruction is displayed in the first window, and the fourth waiting information is displayed in the fourth response dialog box, the fourth waiting information is used to prompt the user that the electronic device is in the process of acquiring the second response result corresponding to the second instruction; after the second response result is acquired, the second response result is displayed in the fourth response dialog box. By adopting the implementation manner, if the previous instruction of the electronic device fails to be sent, the previous instruction is no longer supported to be resent in the case that the user sends a new instruction, and the electronic device stops displaying the second error button, thereby improving the visual experience of the user.

[0046] In an implementation manner, in response to sending the second target content on the first page to the virtual assistant, the second target content is displayed in the first window, and after the at least one third update control associated with the second target content is displayed in the first window, the implementation manner further includes: in the case that the second target content is a picture, in response to a click operation on the picture, a first zoom-in page is displayed, the picture in a zoom-in state is displayed on the first zoom-in page, and a first edit control is displayed on the first zoom-in page; in response to a click operation on the first edit control, an edit main page is displayed, the picture is displayed on the edit main page, and at least one first edit tool is displayed on the edit main page, the first edit tool is used for image processing on the picture. By adopting the implementation manner, the virtual assistant does not need to jump to the third-party application to enable the user to edit the picture based on a manual operation, but automatically accesses the third-party application based on the first control to edit the picture. In this way, the use experience of the user is improved.

[0047] In an implementation manner, in response to a click operation on the picture, a first zoom-in page is displayed, the picture in a zoom-in state is displayed on the first zoom-in page, and a first edit control is displayed on the first zoom-in page, the implementation manner further includes: in response to a double-click operation on the picture in the zoom-in state, or in response to a double-finger pinch operation on the picture in the zoom-in state, the picture in the zoom-in state is zoomed. By adopting the implementation manner, the virtual assistant provides a view large picture function for the initial picture input, and the function supports gesture operation. In this way, the use experience of the user is improved.

[0048] In an implementation manner, in response to the sending of the second target content on the first page to the virtual assistant, the displaying of the second target content in the first window, and the displaying of the at least one first control associated with the second target content in the first window, the method further includes: in response to a click operation on a third target control in the third update control, displaying a third target service recommendation corresponding to the third target control in the first window, the third target service recommendation being used for image processing of the picture; displaying a fifth response dialog box corresponding to the third service recommendation in the first window, and displaying a first processed picture corresponding to the picture in the fifth response dialog box, displaying a second edit control in a region adjacent to the first processed picture in the fifth response dialog box, and displaying a third edit control in a region adjacent to the fifth response dialog box, wherein the second edit control is used for linking to an edit subpage corresponding to the third service recommendation, and the third edit control is used for linking to an edit main page, and the edit main page includes at least one first edit tool used for image processing of the first processed picture. With the implementation manner, the virtual assistant provides the second edit control and the third edit control, so that the user can quickly enter a required edit page to edit the picture.

[0049] In an implementation manner, in response to the sending of the second target content on the first page to the virtual assistant, the displaying of the second target content in the first window, and the displaying of the at least one first control associated with the second target content in the first window, the method further includes: in response to a click operation on a third target control in the third update control, displaying a third target service recommendation corresponding to the third target control in the first window, the third target service recommendation being used for image processing of the picture; displaying a fifth response dialog box corresponding to the third service recommendation in the first window, and displaying a first processed picture corresponding to the picture in the fifth response dialog box, displaying a second edit control in a region adjacent to the first processed picture in the fifth response dialog box, and displaying a third edit control in a region adjacent to the fifth response dialog box, wherein the second edit control is used for linking to an edit subpage corresponding to the third service recommendation, and the third edit control is used for linking to an edit main page, and the edit main page includes at least one first edit tool used for image processing of the first processed picture. With the implementation manner, the virtual assistant provides the second edit control and the third edit control, so that the user can quickly enter a required edit page to edit the picture.

[0050] In an implementation, in response to the click operation on the first processed picture, the second zoom-in page is displayed, and the first processed picture in the zoom-in state is displayed on the second zoom-in page, and after the fourth editing control is displayed on the second zoom-in page, the first processed picture in the zoom-in state is scaled in response to a double-click operation on the first processed picture in the zoom-in state, or in response to a double-finger pinch operation on the first processed picture in the zoom-in state. With this implementation, the virtual assistant provides a large picture viewing function for the first processed picture, and the function supports gesture operations. In this way, the user experience is improved.

[0051] In an implementation, in the first window, the fifth reply dialog box corresponding to the third service recommendation is displayed, and the first processed picture corresponding to the picture is displayed in the fifth reply dialog box, and the second editing control is displayed in a region adjacent to the first processed picture in the fifth reply dialog box, and the third editing control is displayed in a region adjacent to the fifth reply dialog box, and after the first confirmation button is displayed on the edit sub-page in response to the click operation on the second editing control, the first editing operation is performed on the first processed picture on the edit sub-page to obtain a second processed picture, and the first storage button is displayed on the edit main page in response to the click operation on the first confirmation button, and the third processed picture is obtained after the second editing operation is performed on the second processed picture on the edit main page, and the third processed picture is stored in response to the click operation on the first storage button. With this implementation, the user can quickly enter the required edit sub-page based on the second editing control, and the electronic device can also switch between the edit sub-page and the edit main page to meet the user's editing needs, so that the user experience is improved.

[0052] In an implementation, in the first window, the fifth reply dialog box corresponding to the third service recommendation is displayed, and the first processed picture corresponding to the picture is displayed in the fifth reply dialog box, and the second editing control is displayed in a region adjacent to the first processed picture in the fifth reply dialog box, and the third editing control is displayed in a region adjacent to the fifth reply dialog box, and after the first confirmation button is displayed on the edit sub-page in response to the click operation on the second editing control, the first editing operation is performed on the first processed picture on the edit sub-page to obtain a second processed picture, and the first storage button is displayed on the edit main page in response to the click operation on the first confirmation button, and the third processed picture is obtained after the second editing operation is performed on the second processed picture on the edit main page, and the third processed picture is stored in response to the click operation on the first storage button. With this implementation, the user can quickly enter the required edit sub-page based on the second editing control, and the electronic device can also switch between the edit sub-page and the edit main page to meet the user's editing needs, so that the user experience is improved.

[0053] In a second aspect, an embodiment of the present application provides a virtual assistant display apparatus, comprising: a display module, the display module being configured to display a first page; a wake-up module, the wake-up module being configured to wake up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control comprising a service recommendation matched with the first page; a page updating module, the page updating module being configured to update the first page to obtain a first updated page in response to a first operation on the first page; and a control updating module, the control updating module being configured to update the first control based on the first updated page to obtain a first updated control, the first updated control comprising a service recommendation matched with the first updated page.

[0054] The apparatus shown in the embodiments of the present application can provide a service recommendation to a user without inputting an instruction by the user, so that the user can quickly use a service based on the service recommendation, and the virtual assistant can update the service recommendation when the user further performs an operation of updating a page, so that the user can quickly use the service based on the updated service recommendation, and the technical solution shown in the embodiments of the present application improves the user experience.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the memory storing program instructions, when the program instructions are executed by the processor, causing the electronic device to perform the virtual assistant display method provided in the first aspect and any possible implementation manner thereof.

[0056] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium comprising computer instructions, when the computer instructions are run on an electronic device, causing the electronic device to perform the virtual assistant display method provided in the first aspect and any possible implementation manner thereof.

[0057] In a fifth aspect, an embodiment of the present application provides a computer program product, when the computer program product is run on an electronic device, causing the electronic device to perform the virtual assistant display method provided in the first aspect and any possible implementation manner thereof.

[0058] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the third aspect to the fifth aspect can refer to the beneficial effects in the first aspect and any possible implementation manner thereof, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0059] FIG. 1 is a schematic diagram of a scenario in which an electronic device wakes up a virtual assistant;

[0060] FIG. 2 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application;

[0061] FIG. 3 is a schematic diagram of a software structure of an electronic device according to an embodiment of the present application;

[0062] FIG. 4 is a first scenario diagram of an electronic device providing a guide flow according to an embodiment of the disclosure;

[0063] FIG. 5 is a second scenario diagram of an electronic device providing a guide flow according to an embodiment of the disclosure;

[0064] FIG. 6 is a third scenario diagram of an electronic device providing a guide flow according to an embodiment of the disclosure;

[0065] FIG. 7 is a fourth scenario diagram of an electronic device providing a guide flow according to an embodiment of the disclosure;

[0066] FIG. 8 is a fifth scenario diagram of an electronic device providing a guide flow according to an embodiment of the disclosure;

[0067] FIG. 9 is a sixth scenario diagram of an electronic device providing a guide flow according to an embodiment of the disclosure;

[0068] FIG. 10 is a scenario diagram of a first time when a virtual assistant is woken up according to an embodiment of the disclosure;

[0069] FIG. 11 is a scenario diagram of a first time when a window state of a virtual assistant is switched according to an embodiment of the disclosure;

[0070] FIG. 12 is a first scenario diagram of a setting method of a virtual assistant according to an embodiment of the disclosure;

[0071] FIG. 13 is a second scenario diagram of a setting method of a virtual assistant according to an embodiment of the disclosure;

[0072] FIG. 14 is a first wake-up method of a virtual assistant in an application according to an embodiment of the disclosure;

[0073] FIG. 15 is a change method of a navigation bar according to an embodiment of the disclosure;

[0074] FIG. 16 is a second wake-up method of a virtual assistant in an application according to an embodiment of the disclosure;

[0075] FIG. 17 is a third wake-up method of a virtual assistant in an application according to an embodiment of the disclosure;

[0076] FIG. 18 is a fourth wake-up method of a virtual assistant in an application according to an embodiment of the disclosure;

[0077] FIG. 19 is a scenario diagram of an electronic device recognizing a user's intention according to an embodiment of the disclosure;

[0078] FIG. 20 is a fifth wake-up method of a virtual assistant in an application according to an embodiment of the disclosure;

[0079] FIG. 21 is a schematic diagram of a display state of a first window according to an embodiment of the present application;

[0080] FIG. 22 is a schematic diagram of an input state of the first window according to an embodiment of the present application;

[0081] FIG. 23 is a schematic diagram of a display mode of a first control according to an embodiment of the present application;

[0082] FIG. 24 is a schematic diagram of a display mode of a second control according to an embodiment of the present application;

[0083] FIG. 25 is a schematic diagram of an updating process of the first control according to an embodiment of the present application;

[0084] FIG. 26 is a schematic diagram of a scenario of input operation of a user on the first window according to an embodiment of the present application;

[0085] FIG. 27 is a schematic diagram of a first scenario of window state switching of the first window according to an embodiment of the present application;

[0086] FIG. 28 is a schematic diagram of a second scenario of window state switching of the first window according to an embodiment of the present application;

[0087] FIG. 29 is a schematic diagram of a third scenario of window state switching of the first window according to an embodiment of the present application;

[0088] FIG. 30 is a schematic diagram of a fourth scenario of window state switching of the first window according to an embodiment of the present application;

[0089] FIG. 31 is a schematic diagram of a fifth scenario of window state switching of the first window according to an embodiment of the present application;

[0090] FIG. 32 is a schematic diagram of a sixth scenario of window state switching of the first window according to an embodiment of the present application;

[0091] FIG. 33 is a schematic diagram of a seventh scenario of window state switching of the first window according to an embodiment of the present application;

[0092] FIG. 34 is a schematic diagram of a first scenario of moving the first window by an electronic device according to an embodiment of the present application;

[0093] FIG. 35 is a schematic diagram of a second scenario of moving the first window by an electronic device according to an embodiment of the present application;

[0094] FIG. 36 is a schematic diagram of a third scenario of moving the first window by an electronic device according to an embodiment of the present application;

[0095] FIG. 37 is a schematic diagram of a fourth scenario of moving the first window by an electronic device according to an embodiment of the present application;

[0096] FIG. 38 is a fifth scenario diagram illustrating movement of a first window of an electronic device according to an embodiment of the present disclosure;

[0097] FIG. 39 is a first scenario diagram illustrating interaction of a user with a virtual assistant based on a first control according to an embodiment of the present disclosure;

[0098] FIG. 40 is a second scenario diagram illustrating interaction of a user with a virtual assistant based on a first control according to an embodiment of the present disclosure;

[0099] FIG. 41 is a third scenario diagram illustrating interaction of a user with a virtual assistant based on a first control according to an embodiment of the present disclosure;

[0100] FIG. 42 is a first scenario diagram illustrating operation of a first page by a user according to an embodiment of the present disclosure;

[0101] FIG. 43 is a second scenario diagram illustrating operation of a first page by a user according to an embodiment of the present disclosure;

[0102] FIG. 44 is a third scenario diagram illustrating operation of a first page by a user according to an embodiment of the present disclosure;

[0103] FIG. 45 is a fourth scenario diagram illustrating operation of a first page by a user according to an embodiment of the present disclosure;

[0104] FIG. 46 is a first scenario diagram illustrating focus switching according to an embodiment of the present disclosure;

[0105] FIG. 47 is a second scenario diagram illustrating focus switching according to an embodiment of the present disclosure;

[0106] FIG. 48 is a third scenario diagram illustrating focus switching according to an embodiment of the present disclosure;

[0107] FIG. 49 is a fourth scenario diagram illustrating focus switching according to an embodiment of the present disclosure;

[0108] FIG. 50 is a first scenario diagram illustrating ending of a virtual assistant session according to an embodiment of the present disclosure;

[0109] FIG. 51 is a second scenario diagram illustrating ending of a virtual assistant session according to an embodiment of the present disclosure;

[0110] FIG. 52 is a scenario diagram illustrating re-awakening of a virtual assistant according to an embodiment of the present disclosure;

[0111] FIG. 53 is a diagram illustrating a display manner of a first page according to an embodiment of the present disclosure;

[0112] FIG. 54 is a first scenario diagram illustrating sending of a second target content by a user to a virtual assistant according to an embodiment of the present disclosure;

[0113] FIG. 55 is a first interaction diagram of a virtual assistant display method according to an embodiment of the present application;

[0114] FIG. 56 is a scene diagram of a second service plug-in performing recall processing and sorting processing according to an embodiment of the present application;

[0115] FIG. 57 is a first scene diagram of a user interacting with a virtual assistant based on an update control according to an embodiment of the present application;

[0116] FIG. 58 is a first abnormal scene diagram of a user interacting with a virtual assistant according to an embodiment of the present application;

[0117] FIG. 59 is a second abnormal scene diagram of a user interacting with a virtual assistant according to an embodiment of the present application;

[0118] FIG. 60 is a third abnormal scene diagram of a user interacting with a virtual assistant according to an embodiment of the present application;

[0119] FIG. 61 is a second scene diagram of a user sending a second target content to a virtual assistant according to an embodiment of the present application;

[0120] FIG. 62 is a second scene diagram of a user interacting with a virtual assistant based on an update control according to an embodiment of the present application;

[0121] FIG. 63 is a fourth abnormal scene diagram of a user interacting with a virtual assistant according to an embodiment of the present application;

[0122] FIG. 64 is a fifth abnormal scene diagram of a user interacting with a virtual assistant according to an embodiment of the present application;

[0123] FIG. 65 is a sixth abnormal scene diagram of a user interacting with a virtual assistant according to an embodiment of the present application;

[0124] FIG. 66 is a third scene diagram of a user sending a second target content to a virtual assistant according to an embodiment of the present application;

[0125] FIG. 67 is a first scene diagram of a user interacting with a virtual assistant based on a second target content according to an embodiment of the present application;

[0126] FIG. 68 is a third scene diagram of a user interacting with a virtual assistant based on an update control according to an embodiment of the present application;

[0127] FIG. 69 is a scene diagram of a user interacting with a virtual assistant based on a fourth edit control according to an embodiment of the present application;

[0128] FIG. 70 is a scene diagram of a user interacting with a virtual assistant based on a second edit control according to an embodiment of the present application;

[0129] FIG. 71 is a schematic diagram of a scenario in which a user interacts with a virtual assistant based on a third editing control according to an embodiment of the present application;

[0130] FIG. 72 is a schematic diagram of a fourth scenario in which a user sends second target content to a virtual assistant according to an embodiment of the present application;

[0131] FIG. 73 is a schematic diagram of a fifth scenario in which a user sends second target content to a virtual assistant according to an embodiment of the present application;

[0132] FIG. 74 is a schematic diagram of a sixth scenario in which a user sends second target content to a virtual assistant according to an embodiment of the present application;

[0133] FIG. 75 is a schematic diagram of a seventh scenario in which a user sends second target content to a virtual assistant according to an embodiment of the present application. DETAILED DESCRIPTION

[0134] The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0135] In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in the present application is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, "at least one" means one or more, and "multiple" means two or more. "First", "second", and the like do not limit the quantity and execution order, and "first", "second", and the like do not necessarily mean different.

[0136] It should be noted that in the present application, "exemplary" or "for example" is used to mean example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, "exemplary" or "for example" is used to present the relevant concept in a specific manner.

[0137] A virtual assistant is usually provided in an electronic device, which is an interactive service realized by artificial intelligence technology and can simulate the role of a human assistant to quickly handle daily affairs. The virtual assistant includes a voice assistant and a large model assistant, etc. The voice assistant can understand and respond to user voice instructions using voice recognition technology, and provide functions such as information query, schedule management, and device control. The large model assistant can handle complex tasks and data, and provide functions such as text generation, image recognition, and data analysis. These types of assistants can meet different interactive needs of users. The virtual assistant involved in the present application can be any type of virtual assistant, which is not limited in the present application.

[0138] Currently, the virtual assistant cannot provide service recommendations to the user without inputting instructions, so the user cannot quickly use the service based on the service recommendations. The lack of interaction convenience between the user and the virtual assistant affects the user's experience.

[0139] The virtual assistant display method provided in the embodiments of the present application can provide service recommendations to the user without inputting instructions, so that the user can quickly use the service based on the service recommendations. When the user further performs an operation (for example, an operation of updating a page, or an operation of inputting multi-modal content to the virtual assistant), the virtual assistant can update the service recommendations, so that the user can quickly use the service based on the updated service recommendations. The technical solutions shown in the present application improve the user's experience.

[0140] The following exemplary description is made for the case where the user performs an operation of updating a page to make the virtual assistant update the service recommendations. This process solves the problem that the virtual assistant only supports single wake-up and cannot be used cooperatively with an application, and improves the user's experience.

[0141] The following describes the problem that the virtual assistant only supports single wake-up and cannot be used cooperatively with an application.

[0142] FIG. 1 is a schematic diagram of a scenario in which an electronic device wakes up a virtual assistant.

[0143] As shown in A of FIG. 1, an example is illustrated that a user is in a chatting scenario. During the process that the user chats in the chatting interface 2 of the electronic device 1 based on a chatting application, the user can perform a pressing operation on the power key 3. As shown in B and C of FIG. 1, in response to the pressing operation of the user on the power key 3, the electronic device 1 can wake up the virtual assistant at the bottom of the chatting interface 2 to display a first animation 4 corresponding to the virtual assistant. The user can input a voice instruction based on the first animation 4 to make the virtual assistant perform the operation in its program based on the voice instruction, for example, perform the operation of querying a route. As shown in D of FIG. 1, if the user operates the chatting interface 2 again, the virtual assistant ends the session and returns to the chatting interface 2 as shown in A of FIG. 1.

[0144] Correspondingly, in other types of application programs, if the virtual assistant is woken up and the application program is operated again, the virtual assistant ends the session.

[0145] That is to say, at present, the virtual assistant only supports single wake-up and performs the operation in its program, and the application program and the virtual assistant do not support cooperative use, which affects the user experience.

[0146] The virtual assistant display method provided by the embodiments of the present application solves the above problems.

[0147] The virtual assistant display method provided by the embodiments of the present application is described below.

[0148] The virtual assistant display method provided by the embodiments of the present application can be applied to an electronic device. The electronic device includes but is not limited to a mobile phone, a tablet computer, a personal computer, a workstation device, a large-screen device (for example, a smart screen, a smart television, etc.), a wearable device (for example, a smart bracelet, a smart watch), a palm game console, a home game console, a virtual reality device, an augmented reality device, a mixed reality device, a vehicle-mounted intelligent terminal, etc. The embodiments of the present application do not limit the specific technology and specific device form of the electronic device.

[0149] FIG. 2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiments of the present application.

[0150] As shown in FIG. 2, the electronic device 900 can include a processor 910, a memory 920, a Universal Serial Bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 01, an antenna 02, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, a headset jack 970D, a sensor module 980, a key 990, a motor 991, a camera 992, a display screen 993, and a Subscriber Identification module (SIM) card interface 994, etc. Among them, the sensor module 980 can include a touch sensor 980A, a gyroscope sensor 980B, a barometric sensor 980C, a geomagnetic sensor 980D, an acceleration sensor 980E, a distance sensor 980F, a proximity light sensor 980G, a fingerprint sensor 980H, a temperature sensor 980J, etc. Among them, the gyroscope sensor 980B, the barometric sensor 980C, the geomagnetic sensor 980D, the acceleration sensor 980E, etc. can be used to detect the motion state of the electronic device, and therefore, can also be referred to as a motion sensor.

[0151] The processor 910 can include one or more processing units, for example: the processor 910 can include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a Neural-network Processing Unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0152] The memory 920 can be used to store computer executable program codes, and the executable program codes include instructions.

[0153] The USB interface 930 is an interface conforming to the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc.

[0154] The charging management module 940 is used to receive charging input from a charger.

[0155] The power management module 941 is configured to connect the battery 942, and the charging management module 940 is connected to the processor 910.

[0156] The wireless communication function of the electronic device 900 can be implemented by the antenna 01, the antenna 02, the mobile communication module 950, the wireless communication module 960, the modem processor, and the baseband processor.

[0157] The mobile communication module 950 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 900.

[0158] The wireless communication module 960 can provide a solution for wireless communication applied to the electronic device 900.

[0159] The wireless communication module 960 can be one or more devices integrated with at least one communication processing module. The wireless communication module 960 receives electromagnetic waves via the antenna 02, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 910. The wireless communication module 960 can also receive signals to be sent from the processor 910, perform frequency modulation, amplification, and convert the signals to electromagnetic wave radiation via the antenna 02.

[0160] In some embodiments, the antenna 01 of the electronic device 900 is coupled to the mobile communication module 950, and the antenna 02 is coupled to the wireless communication module 960, so that the electronic device 900 can communicate with the network and other devices through wireless communication technology.

[0161] In the embodiments of the present application, the electronic device 900 needs to be in the state of opening the mobile network switch and turning on the WLAN to realize the normal interaction of the virtual assistant.

[0162] The electronic device 900 realizes the display function through the GPU, the display screen 993, and the application processor.

[0163] In the embodiments of the present application, the electronic device 900 can process and display the first window, the first control, and the first page through the GPU. The display screen 993 can display the first window, the first control, and the first page.

[0164] The display screen 993 is configured to display images, videos, etc. The display screen 993 includes a display panel. In some embodiments, the electronic device 900 can include one or N display screens 993, and N is a positive integer greater than 1.

[0165] In the embodiments of the present application, the electronic device 900 can be a foldable electronic device. When the electronic device 900 is a foldable electronic device, there are at least two display screens 993.

[0166] The electronic device 900 can implement a photographing function through an ISP, the camera 992, a video codec, a GPU, a display 993, and an application processor, etc.

[0167] The ISP is configured to process data fed back by the camera 992.

[0168] The camera 992 is configured to capture still images or videos. In some embodiments, the electronic device 900 can include one or N cameras 992, where N is a positive integer greater than 1.

[0169] The electronic device 900 can implement an audio function through an audio module 970, a speaker 970A, a microphone 970B, a microphone 970C, an earphone interface 970D, and an application processor, etc. For example, music playing, voice recording, etc.

[0170] In embodiments of the present application, the electronic device 900 can implement the wake-up of the virtual assistant through the audio module 970.

[0171] The touch sensor 980A is also referred to as a "touch device".

[0172] In embodiments of the present application, the touch sensor 980A can detect a click operation, a long press operation, a drag operation, etc. of a user.

[0173] The gyroscope sensor 980B can be configured to determine the motion posture of the electronic device 900.

[0174] The barometric sensor 980C is configured to measure air pressure.

[0175] The geomagnetic sensor 980D includes a Hall sensor.

[0176] The acceleration sensor 980E can detect the magnitude of acceleration of the electronic device 900 in various directions (generally three axes).

[0177] The distance sensor 980F is configured to measure distance.

[0178] The proximity light sensor 980G can include, for example, a light emitting diode and a light detector, such as a photodiode.

[0179] The fingerprint sensor 980H is configured to collect a fingerprint.

[0180] The temperature sensor 980J is configured to detect temperature.

[0181] The keys 990 include a power key, a volume key, etc. The keys 990 can be mechanical keys. They can also be touch keys. The electronic device 900 can receive key input, and generate key signal input related to user settings and function control of the electronic device 900.

[0182] In the embodiments of this application, the key 990 can be used to wake up the virtual assistant.

[0183] The motor 991 can generate a vibration prompt.

[0184] In the embodiments of this application, the motor 991 can provide vibration feedback to the user when the user performs an interactive operation, thereby improving the user's use experience.

[0185] The SIM card interface 994 is used to connect a SIM card.

[0186] The software system of the electronic device 900 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of this application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 900.

[0187] FIG. 3 is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0188] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer.

[0189] The application layer can include a series of application packages.

[0190] As shown in FIG. 3, the application packages can include battery management, camera, gallery, calendar, call, map, navigation, music, video, short message, etc.

[0191] In the embodiments of this application, the first application can be any type of application described above.

[0192] The application framework layer provides the application layer with application programming interfaces (APIs) and programming frameworks. The application framework layer includes some pre-defined functions.

[0193] As shown in FIG. 3, the application framework layer can include a window manager, an input manager InputManager, a sensor manager SensorManager, a phone manager, a resource manager, a notification manager, etc.

[0194] The input manager can be used to listen to the input events of the user, such as the click event, the sliding event, etc. performed by the user's finger on the display screen 993 of the electronic device 900. By listening to the input events, the electronic device 900 can determine whether the electronic device is being used.

[0195] The sensor manager is configured to listen to data returned by various sensors in the electronic device, such as motion sensor data, proximity light sensor data, temperature sensor data, etc. With the data returned by the various sensors, the electronic device can determine whether it is shaking, or whether the display 993 is blocked, etc.

[0196] The Android Runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0197] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0198] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection, etc.

[0199] The system library can include a plurality of functional modules. For example: a surface manager, media libraries, a three-dimensional graphics processing library (for example: OpenGL ES), a 2D graphics engine (for example: SGL), etc.

[0200] The surface manager is configured to manage the display subsystem, and provides a fusion of 2D and 3D layers for a plurality of applications.

[0201] The media library supports a plurality of commonly used audio, video format playback and recording, and static image files, etc. The media library can support a plurality of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0202] The three-dimensional graphics processing library is configured to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0203] The 2D graphics engine is a drawing engine for 2D drawing.

[0204] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, a power supply driver, an audio driver, and a sensor driver.

[0205] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 900. In other embodiments of the present application, the electronic device 900 can include more or fewer components than the diagram, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0206] After the electronic device 900 is powered on, a guide flow can be provided to the user to enable the user to learn the operation mode of the virtual assistant and complete the initial setting of the virtual assistant.

[0207] It should be noted here that the guide flow can be provided when the electronic device 900 is first enabled, or when the virtual assistant is triggered again after the electronic device 900 is disabled for a period of time. The specific manner in which the electronic device 900 provides the guide flow is not limited by the embodiments of the present application.

[0208] FIG. 4 is a schematic diagram of a first scenario in which the electronic device provides a guide flow according to an embodiment of the present application.

[0209] As shown in FIG. 4, an example is taken in the case of the first start of the electronic device 900. After the electronic device 900 is powered on, a first guide page 10 is displayed. The first guide page 10 can include a virtual assistant enablement control 11, which is used to enable the virtual assistant. The virtual assistant enablement control 11 can include a first option control 12.

[0210] The first option control 12 can be in a selected mode by default, so that the electronic device 900 can enable the virtual assistant. If the first option control 12 is not in the selected mode, the user can click the first option control 12 to select it to enable the virtual assistant.

[0211] The first guide page 10 can also include a first description area 13 for explaining the matters needing attention for enabling the virtual assistant.

[0212] In an implementation, the first description area 13 can include an out of box experience (OOBE) description. The content of the OOBE description can include, for example, "The virtual assistant and its underlying services (fast application engine) need to be connected to the network, need to turn on local recording, read audio, video, make a phone call, send a short message, read a short message, read a unique identifiable setting, read location information, read phonebook data, read user's local number, read calendar, read application list, read call record, turn on mobile network switch, turn on WLAN, modify calendar, modify short message, turn on system notification, need to collect your biometric information, this device only collects personal information related to the above behaviors when using specific business functions, this service provides the user with intelligent voice interaction basic business functions, and also provides more extended business functions, closing the extended business does not affect the use of the basic business functions, enabling the service means that the user agrees to the virtual assistant user agreement, permission use instruction, and knows the statement about the virtual assistant and privacy." The specific content of the OOBE description is not limited in the embodiments of the present application.

[0213] The first guide page 10 further includes a next step button 14. The user can click the next step button 14, and in response to the clicking operation of the user on the next step button 14, the electronic device 900 enters a second guide page 15.

[0214] The second guide page 15 can include a second description area 16, which is used to describe the characteristics of the virtual assistant.

[0215] For example, the characteristic description content in the second description area 16 can include "long press the navigation prompt bar or the home screen button to call the virtual assistant at any time, and any text, picture, or file on the screen can be dragged to the virtual assistant to complete the corresponding question and answer, service provision, etc." The characteristic description content of the virtual assistant is not limited in the embodiments of the present application.

[0216] The second guide page 15 further includes a confirmation button 17. The user can click the confirmation button 17, and in response to the clicking operation of the user on the confirmation button 17, the electronic device 900 enters a third guide page.

[0217] In the above scenario, the electronic device 900 exemplarily describes the matters needing attention and characteristics of the virtual assistant in the guide flow.

[0218] Further, the electronic device 900 provides operation teaching to the user in the guide flow.

[0219] FIG. 5 is a second scenario diagram of an electronic device providing a guide flow according to an embodiment of the present application.

[0220] As shown in FIG. 5, the electronic device 900 can enter a third guide page 18, and the third guide page 18 can include a third description area 19 for describing the function of the virtual assistant.

[0221] For example, the function description content of the third description area 19 can include "wake up the smart voice, search the encyclopedia immediately, make a phone call, or set an alarm, etc.". The function description content of the virtual assistant is not limited by the embodiments of the present application.

[0222] The third guide page 18 further includes a confirmation button 20. The user can click the confirmation button 20, and in response to the clicking operation of the user on the confirmation button 20, the electronic device 900 enters a fourth guide page 21.

[0223] The fourth guide page 21 can include a first teaching area 22. The first teaching area 22 can be used to make the user learn the way of waking up the virtual assistant based on breath.

[0224] The first teaching area 22 can include a first image example area 23 and a first text example area 24. The first image example area 23 can make the user learn the way of waking up the virtual assistant based on breath through images. The first text example area 24 can make the user learn the way of waking up the virtual assistant based on breath through text.

[0225] For example, the first image example area 23 can include an image for representing that the user makes voice input to the bottom microphone of the electronic device 900.

[0226] For example, the text teaching content of the first text example area 24 can include "put the bottom of the electronic device close to the mouth (within 3 cm) and directly speak the voice instruction to the microphone."

[0227] The fourth guide page 21 further includes a next step button 25. The user can click the next step button 25, and in response to the clicking operation of the user on the next step button 25, the electronic device 900 enters a fifth guide page 26.

[0228] The fifth guide page 26 can include a second teaching area 27, and the second teaching area 27 can make the user learn the way of waking up the virtual assistant based on the power key.

[0229] The second teaching area 27 can include a second image example area 28 and a second text example area 29. The second image example area 28 can make the user learn the way of waking up the virtual assistant based on the power key through images. The second text example area 29 can make the user learn the way of waking up the virtual assistant based on the power key through text.

[0230] For example, the second image example area 28 can include an image for representing that the user wakes up the virtual assistant by long pressing the power key for 1 second.

[0231] For example, the text teaching content of the second text example area 29 can include "long press the power key for 1 second to wake up the virtual assistant, and then release the power key to speak; long press the power key for 3 seconds to shut down."

[0232] The fifth guide interface 26 further includes a next step button 30. The user can click the next step button 30, and in response to the clicking operation of the user on the next step button 30, the electronic device 900 enters a sixth guide page 31.

[0233] The sixth guide page 31 can include a first entry area 32 and a first entry animation 33. The first entry area 32 can guide the user to enter a voice instruction to make the virtual assistant record the voiceprint of the user.

[0234] For example, the guided entry content of the first entry area 32 can include "the electronic device is 30 centimeters away from the face, keep the environment quiet, and clearly speak the instruction."

[0235] The user can perform a corresponding voice entry operation according to the guided entry content, and in the process of the voice entry operation of the user, the electronic device 900 can display the first entry animation 33 as a jumping animation effect to indicate that the voice input is currently being received. It should be noted that the first entry animation 33 can also be displayed as a flashing animation effect, and the type of the animation effect of the first entry animation 33 is not limited in the embodiments of the present application.

[0236] After the electronic device 900 completes recording the voiceprint of the user, the electronic device 900 can jump to a first trial page.

[0237] In the above scenario, the electronic device 900 exemplarily illustrates the wake-up mode related to the virtual assistant in the guide flow.

[0238] Further, the electronic device 900 provides the first trial teaching to the user in the guide flow.

[0239] FIG. 6 is a third scenario diagram of the electronic device providing a guide flow according to an embodiment of the present application.

[0240] As shown in FIG. 6, the electronic device 900 can enter a first trial page 34, and the first trial page 34 can include a trial button 35.

[0241] The user can click the trial button 35, and in response to the clicking operation of the user on the trial button 35, the electronic device 900 enters a second trial page 36.

[0242] The electronic device 900 can display a navigation prompt bar 37 at the bottom of the second trial page 36, and display a first prompt box 38 at the top of the second trial page 36, the first prompt box 38 being used to prompt a user of a manner of waking up a virtual assistant.

[0243] For example, the prompt content of the first prompt box 38 can include “long press the bottom navigation bar to wake up the virtual assistant”.

[0244] The user can long press the navigation prompt bar 37, and in response to the long press operation of the user on the navigation prompt bar 37, the electronic device 900 hides the navigation prompt bar 37 at the bottom of the second trial page 36, and displays a first trial window 39 corresponding to the virtual assistant, and at least one first trial control 40 adjacent to the first trial window 39. The first trial control 40 is used to provide a service recommendation matched with the second trial page 36. The service recommendation may, for example, be a full-text translation, a summary abstract, and the like. The service recommendation is specifically described in subsequent embodiments of the present application, and will not be described here.

[0245] It should be noted here that the first trial window 39 is displayed in a default state when it is first woken up, and the type of window state corresponding to the virtual assistant is specifically described in subsequent embodiments of the present application, and will not be described here.

[0246] The electronic device 900 can also update the first prompt box 38 at the top of the second trial page 36 to generate a second prompt box 41, the second prompt box 41 being used to prompt the user that the virtual assistant has been woken up.

[0247] For example, the prompt content of the second prompt box 41 can include “the virtual assistant has been quickly woken up to understand the content”.

[0248] After the second prompt box 41 is displayed for a certain time length, the electronic device 900 can update the second prompt box 41 at the second trial page 36 to generate a third prompt box 42, the third prompt box 42 being used to prompt the user of a manner of exiting the virtual assistant. For example, the display time length of the second prompt box 41 can be 2 seconds.

[0249] It should be noted here that the manner in which the electronic device 900 updates the second prompt box 41 at the second trial page 36 to generate the third prompt box 42 can be implemented based on a transition effect. The specific update manner of the page content is not limited in the embodiments of the present application.

[0250] For example, the prompt content of the third prompt box 42 can include “slide down from the top window edge of the first trial window to manually exit the virtual assistant”.

[0251] The user can perform a downward swipe operation from the top window edge of the first trial window 39 to end the session with the virtual assistant. Correspondingly, the electronic device 900 updates the third prompt box 42 at the top of the second trial page 36 and generates a fourth prompt box 43, which is used to prompt the user that the virtual assistant has been manually exited.

[0252] For example, the prompt content of the fourth prompt box 43 can include “The virtual assistant has been manually exited.”

[0253] In the above scenario, the electronic device 900 provides trial teaching of gesture navigation to the user in the guide flow. It should be noted that the present embodiment does not provide trial teaching of all operation modes, and the trial teaching of other modes can be adapted based on the specific operation mode, for example, the trial teaching of the navigation bar can be individually adapted based on the specific operation mode of the navigation bar, and the specific mode of the trial teaching is not limited in the present embodiment.

[0254] Further, the electronic device 900 provides second trial teaching to the user in the guide flow.

[0255] FIG. 7 is a fourth scenario diagram of the electronic device providing a guide flow according to an embodiment of the present application.

[0256] As shown in FIG. 7, the electronic device 900 can enter a third trial page 44 and display a fifth prompt box 45 at the top of the third trial page 44 and a navigation prompt bar 37 at the bottom of the third trial page 44, wherein the fifth prompt box 45 is used to prompt the user of the way to wake up the virtual assistant.

[0257] For example, the third trial page 44 can be a page in a document application, for example, which can provide a plurality of documents. For example, a first document 46 is included. The prompt content of the fifth prompt box 45 can include “Please long press and drag the first document to the bottom navigation bar.”

[0258] The user can long press the first document 46 and perform a drag operation on the first document 46 to the navigation prompt bar 37. In response to the user's operation of dragging the first document 46 to the navigation prompt bar 37, the electronic device 900 displays a first trial window 39 corresponding to the virtual assistant at the bottom of the third trial page 44 and at least one first trial control 40 adjacent to the first trial window 39.

[0259] It should be noted that the first trial window 39 has different state types. In the current scenario, the first trial window 39 is in a state capable of containing interactive content, which is different from the default state displayed in the foregoing embodiments. The state types of the window corresponding to the virtual assistant are described in subsequent embodiments of the present application, and the present embodiment will not be described here.

[0260] Furthermore, the electronic device can also update the fifth prompt box 45 at the top of the third trial page 44 and generate a sixth prompt box 47, which is used to prompt the user that the virtual assistant has been activated.

[0261] For example, the message in the sixth prompt box 47 could include "The virtual assistant has been quickly activated for content understanding."

[0262] In the above scenario, the electronic device 900 provides users with a trial tutorial during the guidance process, showing them how to drag page content to the navigation bar 37 to wake up the virtual assistant without waking it up.

[0263] Furthermore, the electronic device 900 provides users with a third type of trial tutorial during the boot process.

[0264] Figure 8 is a schematic diagram of the fifth scenario of the electronic device providing the boot process according to the embodiments of this application.

[0265] As shown in Figures 7 and 8, after the sixth prompt box 47 has been displayed for a certain period of time, the electronic device 900 can restore the first trial window 39 from a state capable of accommodating interactive content to its default state on the third trial page 44, and update the sixth prompt box 47 at the top of the third trial page 44, generating a seventh prompt box 48. The seventh prompt box 48 is used to prompt the user to long-press and drag the first document 46 to the virtual assistant. The display duration of the sixth prompt box 47 can be, for example, 2 seconds.

[0266] It should be noted that the way the electronic device 900 restores the first trial window 39 from a state capable of accommodating interactive content to its default state on the third trial page 44, and updates the sixth prompt box 47 at the top of the third trial page 44 to generate the seventh prompt box 48, can be achieved based on a transition effect. This application embodiment does not limit the specific method of updating page content.

[0267] For example, the prompt content of the seventh prompt box 48 may include "Please long press and drag the first document to the virtual assistant".

[0268] The user can long press the first document 46 and drag the first document 46 to the first trial window 39. In response to the user dragging the first document 46 to the first trial window 39, the electronic device 900 displays the first trial window 39 from the default state to a state that can accommodate interactive content, and displays at least one first trial control 40 adjacent to the first trial window 39.

[0269] Furthermore, the electronic device can also update the seventh prompt box 48 at the top of the third trial page 44 and generate the eighth prompt box 49, which is used to prompt the user that the trial tutorial has been completed.

[0270] For example, the prompt content of the eighth prompt box 49 can include "all operations have been completed."

[0271] In the above scenario, the electronic device 900 provides the user with a trial teaching of dragging the page content to the virtual assistant in the case of waking up the virtual assistant, and the virtual assistant providing the corresponding service in the boot flow.

[0272] Further, the electronic device 900 provides a flow summary in the boot flow.

[0273] FIG. 9 is a sixth scenario diagram of the electronic device providing a boot flow according to an embodiment of the present application.

[0274] As shown in FIGS. 8 and 9, after the eighth prompt box 49 is displayed for a certain time length, the electronic device 900 can display a first summary page 50, and display a first summary area 51 in the first summary page 50, where the first summary area 51 is used to summarize the functions of the virtual assistant.

[0275] For example, the function summary content of the first summary area 51 can include "understanding screen content, automating execution, and intelligent agent store". The function summary content of the virtual assistant is not limited in the embodiments of the present application.

[0276] It should be noted that any page of the above boot flow can be provided with an exit button, so that the user can end the boot flow in any process, and any page can be provided with a previous step button, so that the user can return to the previous page.

[0277] It should be noted that in any page of the above boot flow, when the first trial window 39 is displayed, the electronic device 900 displays the same animation effect (hereinafter referred to as animation effect) as in the actual operation process on the first trial window 39. The display mode of the animation effect is not specifically described in the above embodiments, and will be described in detail in subsequent embodiments of the present application.

[0278] Any page of the boot flow provided by the embodiments of the present application can add or delete page elements according to actual conditions, and the specific display content of the boot page is not limited in the embodiments of the present application.

[0279] After the user completes the boot flow, the electronic device 900 can use the virtual assistant.

[0280] FIG. 10 is a scenario diagram of the first time waking up the virtual assistant according to an embodiment of the present application.

[0281] As shown in FIG. 10, the user can wake up the virtual assistant in various manners, such as the voice-based manner, the power key-based manner, the navigation bar-based manner, and the drag operation-based manner, as shown in the guide flow. In addition to the manners in the examples, the Bluetooth key-based manner and the custom gesture operation-based manner can also be used to wake up the virtual assistant. The embodiments of the present application do not limit the manners of waking up the virtual assistant.

[0282] The power key 52-based manner is exemplarily described.

[0283] The user can long press the power key 52 to wake up the virtual assistant for the first time. In order to ensure that the user understands the usage manner of the virtual assistant, the electronic device 900 displays the first navigation page 53 and the first navigation area 54 in the first navigation page 53 in response to the user long pressing the power key 52 to wake up the virtual assistant for the first time, where the first navigation area 54 is used to describe the features of the virtual assistant.

[0284] For example, the feature description content in the first navigation area 54 can include "long press the navigation prompt bar or the home button to wake up the virtual assistant at any time, and any text, picture, or file on the screen can be dragged to the virtual assistant to complete the corresponding question and answer, service provision, etc.". The embodiments of the present application do not limit the feature description content of the virtual assistant.

[0285] The first navigation page 53 further includes a trial button 55 and a quit button 56. The user can click the trial button 55, and the electronic device 900 enters the guide flow provided in the foregoing embodiments in response to the user clicking the trial button 55. It should be noted that after the user enters the guide flow, whether the user interrupts the guide flow or not, the electronic device 900 no longer displays the first navigation page 53 when the virtual assistant is woken up again.

[0286] The user can click the quit button 56, and the electronic device 900 closes the first navigation page 53 and displays the first window corresponding to the virtual assistant in response to the user clicking the quit button 56. It should be noted that the quit button 56 has different display forms, for example, the text description corresponding to the quit button 56 can be "quit" or "I know". In this way, the language habits of the user can be adapted. Although different language description manners can be used, the quit button 56 with different language description manners all perform the function of closing the first navigation page 53.

[0287] If the electronic device 900 closes the first navigation page 53, it indicates that the user does not enter the guide flow based on the actual operation, and then the electronic device 900 can display the first navigation page 53 again in the process of the user subsequently waking up the virtual assistant. The embodiment of the present application can set the number of times of displaying the first navigation page 53, for example, the first navigation page 53 can be displayed only once in a single day, and the total number of times of displaying the first navigation page 53 can be three times. The display frequency of the first navigation page 53 can be set according to the actual situation, and the embodiment of the present application does not limit this.

[0288] In an implementation manner, the electronic device 900 can skip the step of displaying the first navigation page 53 when the virtual assistant is woken up for the first time based on the system setting.

[0289] It should be noted that, in addition to the ways of waking up the virtual assistant based on the voice and the Bluetooth key, the ways of waking up the virtual assistant for the first time of the remaining virtual assistants can refer to the way of waking up the virtual assistant for the first time based on the power key 52, and the embodiment of the present application does not repeat the description.

[0290] In this way, the user can use the virtual assistant after waking up the virtual assistant for the first time and completing the guide.

[0291] In addition, in order to further ensure that the user understands the use mode of the virtual assistant, the electronic device 900 displays the first navigation prompt box to guide the user to move the first window in response to the user triggering the window state switching of the virtual assistant for the first time in the process of the user using the virtual assistant.

[0292] FIG. 11 is a schematic diagram of a scene of triggering the window state switching of the virtual assistant for the first time according to an embodiment of the present application.

[0293] As shown in FIG. 11, the user can wake up the virtual assistant on the home screen page 57 of the electronic device 900 based on the power key 52 to display the first window 58 corresponding to the virtual assistant and at least one first control 59 adjacent to the first window 58, and the first control 59 includes the service recommendation matched with the home screen page 57, for example, “organize the desktop”, “change the wallpaper”, etc.

[0294] When the first window 58 is woken up, the first state window 60 is displayed by default. The embodiment of the present application will be described in detail.

[0295] The first state window 60 includes a navigation bar 61, which can be displayed at the top of the first state window 60.

[0296] The user can input text content in the first status window 60. In response to the user inputting the text content, the electronic device 900 displays the second status window 62, so that the second status window 62 can accommodate the interaction content between the user and the virtual assistant. Subsequent embodiments of the present application will make specific descriptions of the second status window 62.

[0297] In the case where the first status window 60 is switched to the second status window 62, the navigation bar 61 can be displayed at the top of the second status window 62.

[0298] In response to the electronic device 900 displaying the second status window 62 for the first time, the electronic device 900 displays the first navigation prompt box 63 in the area adjacent to the navigation bar 61 in the second status window 62 to guide the user to move the second status window 62.

[0299] For example, the prompt content in the first navigation prompt box 63 can be "Try to press and drag the top bar to move the window position."

[0300] The first navigation prompt box 63 further includes an exit button 64. The user can click the exit button 64. In response to the user clicking the exit button 64, the electronic device 900 stops displaying the first navigation prompt box 63.

[0301] The user can perform a long-pressing and dragging operation on the navigation bar 61 according to the prompt content in the first navigation prompt box 63. In response to the user performing the long-pressing and dragging operation on the navigation bar 61, the electronic device 900 moves the second status window 62 in the home screen page 57 and stops displaying the first navigation prompt box 63.

[0302] The second status window 62 further includes an exit button 65. The user can click the exit button 65. In response to the user clicking the exit button 65, the electronic device 900 ends the session of the virtual assistant and stops displaying the first navigation prompt box 63.

[0303] The user can also end the session of the virtual assistant based on the gesture operation. In response to the user gesture operation, the electronic device 900 ends the session of the virtual assistant and stops displaying the first navigation prompt box 63.

[0304] The embodiments of the present application do not limit the manner in which the first navigation prompt box 63 is stopped from being displayed.

[0305] The embodiments of the present application can set the number of times the first navigation prompt box 63 is displayed, for example, set the number of times the first navigation prompt box 63 is displayed to be displayed only once when the second status window 62 is triggered for the first time. The display frequency of the first navigation prompt box 63 can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0306] After the electronic device 900 is powered on, a default setting mode is usually adopted for the virtual assistant, and the electronic device 900 can also change the related setting mode of the virtual assistant in the setting application software.

[0307] The setting mode of the virtual assistant is described below.

[0308] FIG. 12 is a first scenario diagram of the setting mode of the virtual assistant provided by the embodiment of the application.

[0309] As shown in FIG. 12, the electronic device 900 can display a setting application program 66 in the home screen page 57. The user can perform a click operation on the setting application program 66, and in response to the click operation of the user on the setting application program 66, the electronic device 900 displays a first setting page 67, and the first setting page 67 can include a virtual assistant option 68.

[0310] The user can perform a click operation on the virtual assistant option 68, and in response to the click operation of the user on the virtual assistant option 68, the electronic device 900 enters a second setting page 69, and the second setting page 69 can include a virtual assistant setting option 70.

[0311] The user can perform a click operation on the virtual assistant setting option 70, and in response to the click operation of the user on the virtual assistant setting option 70, the electronic device 900 enters a third setting page 71, and the third setting page 71 can include a system navigation wake-up option 72.

[0312] The user can perform a click operation on the system navigation wake-up option 72, and in response to the click operation of the user on the system navigation wake-up option 72, the electronic device 900 enters a fourth setting page 73, and the fourth setting page 73 can include a system navigation wake-up control 74, and the system navigation wake-up control 74 can be in a default open state, so that the virtual assistant can be woken up through the navigation bar 61 or the home screen button in the navigation bar on any page of the electronic device 900.

[0313] In an implementation manner, if the system navigation wake-up control 74 is set to the closed state, the electronic device 900 does not display the navigation bar 61.

[0314] Since the wake-up mode of the virtual assistant is associated with different system navigation modes, the related setting item of the virtual assistant is also included in the setting page of the system navigation mode.

[0315] FIG. 13 is a second scenario diagram of the setting mode of the virtual assistant provided by the embodiment of the application.

[0316] As shown in FIG. 13, the electronic device 900 can display the settings application 66 in the home screen page 57. The user can perform a click operation on the settings application 66, and in response to the click operation of the user on the settings application 66, the electronic device 900 displays a first settings page 67, and the first settings page 67 can further include a system and update option 75.

[0317] The user can perform a click operation on the system and update option 75, and in response to the click operation of the user on the system and update option 75, the electronic device 900 enters a fifth settings page 76, and the fifth settings page 76 can include a system navigation mode setting option 77.

[0318] The user can perform a click operation on the system navigation mode setting option 77, and in response to the click operation of the user on the system navigation mode setting option 77, the electronic device 900 enters a sixth settings page 78, and the sixth settings page 78 can include a gesture navigation option 79 and a three-key navigation option 80.

[0319] It should be noted here that the three-key navigation is a navigation bar that provides the user with operation buttons, and usually includes a back button, a home button, and a multi-function button. The embodiments of the present application only exemplarily illustrate the navigation bar with the three-key navigation, and in fact, the navigation bar can also contain more buttons, which are not limited by the embodiments of the present application.

[0320] The user can perform a click operation on the gesture navigation option 79, and in response to the click operation of the user on the gesture navigation option 79, the electronic device 900 enters a gesture navigation page 81, and the gesture navigation page 81 can include a display prompt bar control 82 and a system navigation wake-up control 83, both of which can be in a default open state, so that on any page of the electronic device 900, the virtual assistant can be woken up through the navigation bar 61.

[0321] The user can perform a click operation on the three-key navigation option 80, and in response to the click operation of the user on the three-key navigation option 80, the electronic device 900 enters a three-key navigation page 84, and the three-key navigation page 84 can include a system navigation wake-up control 85, which can be in a default open state, so that on any page of the electronic device 900, the virtual assistant can be woken up through the home button of the navigation bar.

[0322] The use process of the virtual assistant is further described below.

[0323] The embodiments of the present application exemplarily illustrate the use process of the virtual assistant in the application.

[0324] FIG. 14 is a schematic diagram of a first wake-up mode of a virtual assistant in an application according to an embodiment of the present application.

[0325] As shown in FIG. 14, the electronic device 900 can display a program icon 86 of a first application, which can be, for example, a mailbox application, in a home screen page 57.

[0326] The following embodiments of the present application exemplarily take the first application as the mailbox application, and the first application can also be a chat application or other types of applications. The specific type of the first application is not limited in the embodiments of the present application.

[0327] The user can perform a click operation on the program icon 86. In response to the click operation of the user on the program icon 86, the electronic device 900 displays a first page 87.

[0328] It should be noted that the first page 87 in the embodiments of the present application can be any application page in the first application. In the subsequent embodiments of the present application, the first page 87 can also be a chat page. The type of the first page 87 is not limited in the embodiments of the present application. After the electronic device 900 starts the first application, the electronic device 900 can display the first page 87 after displaying multiple pages such as a guide page and an entry page in the first application. The process of the electronic device 900 displaying other pages is not described herein.

[0329] The embodiments of the present application are exemplarily described in the manner of gesture navigation of the electronic device 900. In the case of gesture navigation of the electronic device 900, the electronic device 900 displays a navigation bar 61 at the bottom of the display screen.

[0330] The user can perform a long press operation on the navigation bar 61. In response to the long press operation of the user on the navigation bar 61, the electronic device 900 wakes up the virtual assistant on the first page 87 to display a first status window 60 corresponding to the virtual assistant and at least one first control 59 adjacent to the first status window 60, wherein the first control 59 includes a service recommendation matched with the first page 87.

[0331] It should be noted that the user can also perform other types of operations on the navigation bar 61 to wake up the virtual assistant, for example, a sliding operation, a multiple click operation, a custom gesture operation, etc. The embodiments of the present application are exemplarily described in the manner of the long press operation on the navigation bar 61. The type of the gesture operation to wake up the virtual assistant is not limited in the embodiments of the present application.

[0332] In an implementation manner, in order to improve the user experience, the navigation bar 61 can be changed in animation during the long press operation of the user on the navigation bar 61 to indicate that the user is currently performing the long press operation.

[0333] FIG. 15 is a schematic view of a dynamic effect changing mode of the navigation bar according to an embodiment of the present application.

[0334] As shown in FIG. 15, the initial display specification of the navigation bar 61 can be 4x120dp.

[0335] The density-independent pixels (dp) is a pixel unit used for abstracting and standardizing different screen densities.

[0336] During the long-press operation of the navigation bar 61 by the user, the electronic device 900 can change the display specification of the navigation bar 61 from 4x120dp to 4x140dp, and display a first dynamic effect on the navigation bar 61 for a preset dynamic effect duration. The preset dynamic effect duration can be, for example, 1 second, and the first dynamic effect can be a flashing dynamic effect. The specific duration of the preset dynamic effect duration and the dynamic effect type of the first dynamic effect are not limited in the present application.

[0337] Further, during the long-press operation of the navigation bar 61 by the user, the electronic device 900 can provide vibration feedback through a motor to improve the pressing experience of the user.

[0338] The other wake-up modes of the virtual assistant are exemplarily described below.

[0339] FIG. 16 is a schematic view of a second wake-up mode of the virtual assistant in an application according to an embodiment of the present application.

[0340] As shown in FIG. 16, a three-key navigation mode of the electronic device 900 is exemplarily described. In the case where the electronic device 900 adopts the three-key navigation mode, the electronic device 900 displays a navigation bar 88 at the bottom of the display screen thereof, and displays a home screen button 89 in the navigation bar 88.

[0341] The user can perform a long-press operation on the home screen button 89. In response to the long-press operation of the home screen button 89 by the user, the electronic device 900 wakes up the virtual assistant on the first page 87 to display a first status window 60 of the virtual assistant and at least one first control 59 adjacent to the first status window 60, wherein the first control 59 includes a service recommendation matched with the first page 87.

[0342] It should be noted that the user can also perform other types of operations on the home screen button 89 to wake up the virtual assistant, for example, a sliding operation, a multiple-click operation, a custom gesture operation, etc. The present application exemplarily describes the long-press operation on the home screen button 89, and the type of the gesture operation for waking up the virtual assistant is not limited in the present application.

[0343] In an implementation, in order to improve the user experience, during the long-press operation of the home button 89 by the user, the navigation bar 88 and the home button 89 can be changed in a dynamic effect to indicate that the user is currently performing a long-press operation. The specific manner in which the home button 89 is changed in a dynamic effect is not limited in the embodiments of the present application.

[0344] In the first page 87, the electronic device 900 displays the navigation bar 88 and the first status window 60. Therefore, the first status window 60 can avoid the navigation bar 88, for example, by being displayed in the top area of the navigation bar 88.

[0345] It should be noted that, in the subsequent use of the virtual assistant, the first status window 60 can be switched to a window in another state, and the first window 58 in any state can avoid the navigation bar 88.

[0346] FIG. 17 is a schematic diagram of a third wake-up manner of a virtual assistant in an application according to an embodiment of the present application.

[0347] As shown in FIG. 17, whether the electronic device 900 adopts a gesture navigation manner or a three-key navigation manner, the electronic device 900 can wake up the virtual assistant based on a key. For example, the key is the power key 52.

[0348] The user can perform a long-press operation on the power key 52. In response to the long-press operation of the power key 52 by the user, the electronic device 900 wakes up the virtual assistant in the first page 87 to display the first status window 60 corresponding to the virtual assistant and at least one first control 59 adjacent to the first status window 60, and the first control 59 includes a service recommendation matched with the first page 87.

[0349] It should be noted that the user can also perform other types of operations on the power key 52 to wake up the virtual assistant, for example, a multiple-click operation, and the like. The embodiments of the present application are exemplarily described by taking the long-press operation on the power key 52, and the operation type for waking up the virtual assistant is not limited in the embodiments of the present application.

[0350] FIG. 18 is a schematic diagram of a fourth wake-up manner of a virtual assistant in an application according to an embodiment of the present application.

[0351] As shown in FIG. 18, a gesture navigation manner of the electronic device 900 is exemplarily described. In the case where the electronic device 900 adopts the gesture navigation, the electronic device 900 displays the navigation bar 61 at the bottom of the display screen.

[0352] The user can wake up the virtual assistant in the first page 87 by dragging the first target content 90 of the first page 87 to the navigation bar 61.

[0353] The user can perform a selection operation on the first target content 90 of the first page 87. In response to the selection operation on the first target content 90, the electronic device 900 generates a first selected area 91 at a position where the first target content 90 is located. The first selected area 91 can be highlighted.

[0354] It should be noted that the first target content 90 in the embodiments of the present application can be at least one of a picture, text, a document, and a video. The specific type of the first target content 90 is not limited in the embodiments of the present application.

[0355] The user can perform a long press operation on the first selected area 91. In response to the long press operation of the user on the first selected area 91, the electronic device 900 generates a first moving area 92, which is used to copy and move the first target content 90.

[0356] It should be noted that when the long press operation is performed on the first selected area 91, the navigation bar 61 can change in animation effect to indicate that the user can wake up the virtual assistant at the corresponding position thereof to provide a corresponding service.

[0357] In an implementation manner, during the long press operation of the user on the first selected area 91, the electronic device 900 can change the display specification of the navigation bar 61 from 4x120dp to 4x140dp, and display a first animation effect on the navigation bar 61 at a preset animation effect frequency. The preset animation effect frequency can be 3 times, for example, and the first animation effect can be a flashing animation effect. The specific duration of the preset animation effect duration and the animation effect type of the first animation effect are not limited in the embodiments of the present application.

[0358] The user can drag the first moving area 92 to achieve the user demand. It should be noted that a plurality of services are provided in the electronic device 900. For example, an intent recognition service. The intent recognition service allows the user to achieve information transmission and operation across applications and devices through a drag operation. The user can trigger the intent recognition service by dragging the first moving area 92 to the edge of the screen.

[0359] That is, during the dragging of the first moving area 92 by the user, the electronic device 900 cannot determine that the user's intention is to call the virtual assistant or to call the intent recognition service. Therefore, the electronic device 900 needs to identify the user's intention to call the corresponding service.

[0360] The electronic device 900 can identify the user's intention based on the region corresponding to the drag operation to call the corresponding service.

[0361] FIG. 19 is a scene diagram of the electronic device identifying the user's intention according to an embodiment of the present application.

[0362] As shown in FIGS. 18 and 19, the electronic device 900 can be provided with a first preset area 95. The first preset area 95 can be an area extending a certain height upward from the bottom edge of the display screen, which can account for 10% of the total area of the display screen, for example. The embodiments of the present application do not limit the position and area of the first preset area.

[0363] During the user's dragging of the first moving area 92, in response to the user's dragging operation on the first moving area 92, in the case where the first moving area 92 does not reach the first preset area 95, the electronic device 900 displays a first transition window 93 at the position where the navigation bar 61 is located, and displays a first light effect at the screen edge 94.

[0364] The first transition window 93 can be in the form of a capsule, and the first light effect can have a weak light intensity. The embodiments of the present application do not limit the display form of the first transition window 93 and the light effect intensity of the first light effect.

[0365] In this way, the first transition window 93 and the first light effect at the screen edge 94 simultaneously guide the user, so that the user can choose to perform the corresponding function.

[0366] In this case, the user can drag the first moving area 92 to the first transition window 93, or can drag the first moving area 92 to the screen edge 94.

[0367] That is, based on the setting of the first preset area 95, it is ensured that the electronic device 900 will not mistakenly invoke the virtual assistant before reaching the area, avoiding the mismatch between the services provided by the virtual assistant and the actual intention of the user. It should be noted here that the manner in which the electronic device 900 determines that the first moving area 92 does not reach the first preset area 95 can be achieved based on detecting that the first moving area 92 moves on other preset areas.

[0368] During the user's further dragging of the first moving area 92 to the first transition window 93, in the case where the first moving area 92 reaches the first preset area 95, the electronic device 900 displays a second transition window 96 at the position where the first transition window 93 is located, and stops displaying the first light effect at the screen edge 94.

[0369] In this case, the electronic device 900 defaults that the user needs to invoke the virtual assistant, and closes the intention recognition service, so that the user cannot drag the first moving area 92 to the screen edge 94. In this way, the accuracy of the electronic device 900 in recognizing the user's intention is ensured.

[0370] The second transition window 96 can be in a capsule shape. The capsule shape of the second transition window 96 can be different from the capsule shape of the first transition window 93. For example, the second transition window 96 can have a second dynamic effect on the window edge on the basis of the capsule shape of the first transition window 93. The second dynamic effect can be at least one of a color flashing dynamic effect and a halo dynamic effect. The display shape of the second transition window 96 and the type of the second dynamic effect are not limited in the embodiments of the present application.

[0371] When the first moving area 92 reaches the second transition window 96, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first status window 60 corresponding to the virtual assistant. (The process of displaying the first status window 60 is not shown in the figure.) Further, the user drags the first moving area 92 to the first status window 60. In response to the operation of the user dragging the first moving area 92 into the first status window 60, the electronic device 900 displays the second status window 62 and displays the first target content 90 in the second status window 62.

[0372] In the embodiments, the settings of the first transition window 93 and the second transition window 96 ensure that the electronic device 900 can accurately identify the user's intention and form a coherent visual experience between the first transition window 93, the second transition window 96, the first status window 60, and the second status window 62 to improve the user's visual experience. The electronic device 900 avoids the screen hot area (for example, the screen edge 94 corresponding to the intention recognition service) based on the setting of the first preset area 95.

[0373] The electronic device 900 can also identify the user's intention based on the displacement corresponding to the dragging operation to call the corresponding service. This manner can be implemented by referring to the foregoing embodiments. For example, during the dragging of the first moving area 92 by the user, in response to the dragging operation of the user on the first moving area 92, the electronic device 900 displays the first transition window 93 at the position of the navigation bar 61 and displays the first light effect on the screen edge 94 when the first moving area 92 does not reach the first preset displacement. The specific manner of identifying the user's intention by the electronic device 900 is not limited in the embodiments of the present application.

[0374] The electronic device 900 can also use other manners of identifying the user's intention. The specific manner of identifying the user's intention by the electronic device 900 is not limited in the embodiments of the present application.

[0375] In an implementation manner, the user can perform the multiple selecting and dragging operations on different first target content 90 in the first page 87, and if the user does not wake up the virtual assistant within the preset number of dragging times, it indicates that the selecting and dragging operation of the user is for the intent recognition service. Thus, after the preset number of dragging times is reached, if the user performs the selecting and dragging operation again, the electronic device stops displaying the first transition window 93 and only displays the first light effect at the screen edge 94.

[0376] FIG. 20 is a schematic diagram of a fifth wake-up manner of the virtual assistant in an application according to an embodiment of the present application.

[0377] As shown in FIG. 20, an example is taken to illustrate that the electronic device 900 adopts the three-key navigation mode. In the case where the electronic device 900 adopts the three-key navigation mode, the electronic device 900 displays the navigation bar 88 at the bottom of the display screen thereof and displays the home screen button 89 in the navigation bar 88.

[0378] The user can wake up the virtual assistant on the first page 87 by dragging the first target content 90 of the first page 87 to the home screen button 89.

[0379] The specific process in which the user drags the first target content 90 of the first page 87 to the home screen button 89 can refer to the foregoing embodiments.

[0380] In this embodiment, when the electronic device 900 displays the first transition window 93 and the second transition window 96, the first transition window 93 and the second transition window 96 can be overlaid on the navigation bar 88 or can avoid the navigation bar 88.

[0381] When the electronic device 900 displays the first window 58, the navigation bar 88 can be avoided, for example, the first window 58 is displayed in the top area of the navigation bar 88. The display position of the first window 58 is not limited in the embodiments of the present application.

[0382] Among the fourth wake-up manner and the fifth wake-up manner of the virtual assistant in the application, the manner in which the electronic device 900 recognizes the user's intention is involved. It should be noted that the manner in which the electronic device 900 recognizes the user's intention can be applicable to the horizontal screen mode and the vertical screen mode of the electronic device 900, or can be applicable to the unfolded state and the folded state of the foldable electronic device when the electronic device 900 is a foldable electronic device, and the embodiments of the present application are not limited in this regard.

[0383] The electronic device 900 can set different priorities for the virtual assistant and the intent recognition service in different use modes. For example, in the horizontal screen mode, the response priority of the navigation bar 88 can be lower than the response priority of the intent recognition service. Specifically, in the horizontal screen mode, the navigation bar 88 and the screen edge 94 are both located on the right side of the display screen. At this time, the screen edge 94 is the bottom edge of the display screen in the vertical screen mode. The screen edge 94 responds to the user's drag operation preferentially to make the electronic device 900 execute the intent recognition service. The embodiment of the present application does not limit the priority setting mode of the electronic device 900 in different use modes.

[0384] The electronic device 900 can also have more ways to wake up the virtual assistant, which are not limited by the embodiment of the present application.

[0385] When the electronic device 900 wakes up the virtual assistant, the first window 58 corresponding to the virtual assistant is displayed. The first window 58 has different display states, which are described in detail below.

[0386] FIG. 21 is a schematic diagram of the display state of the first window according to an embodiment of the present application.

[0387] The first window 58 includes a first state window 60, a second state window 62, and a third state window 97.

[0388] As shown in A of FIG. 21, the first state window 60 can include a first interaction area 98, which can include an input bar 99, a first assistant icon 100, a multifunction button 101, and a voice input button 102. The first interaction area 98 is used to receive the input operation of the user.

[0389] The input bar 99 can be used for the user to perform an input operation, the first assistant icon 100 can prompt the user that the current window is used for a conversation with the virtual assistant, the multifunction button 101 can provide a third-party service, and the voice input button 102 can make the user perform a voice input operation.

[0390] It should be noted that other types of window elements can also be set in the first state window 60 according to actual conditions to achieve different functions, and the embodiment of the present application does not limit the specific content set in the first state window 60.

[0391] The first state window 60 can also include a first window dynamic effect 103, which can be a colored frame surrounding the first interaction area 98, or other types of dynamic effects. The embodiment of the present application does not limit the dynamic effect type of the first window dynamic effect 103.

[0392] The first status window 60 can have a first height H1, which usually accounts for a small proportion of the total height of the display screen, since the first status window 60 usually corresponds to the case that the virtual assistant is woken up for the first time and no interaction content is input by the user.

[0393] In the first status window 60, the navigation bar 61 is displayed in the top area of the first status window 60. In the case that the electronic device 900 displays the first status window 60, the first control 59 can be displayed in the area adjacent to the outer periphery of the first status window 60.

[0394] The second status window 62 can include a first sub-window 104 and a second sub-window 105.

[0395] As shown in B of FIG. 21, the first sub-window 104 includes the first interaction area 98 and a first panel 106 displayed in a part of the display screen of the electronic device 900.

[0396] It should be noted here that the first interaction area 98 in the first sub-window 104 can be the same as the first interaction area 98 in the first status window 60. The subsequent embodiments of the present application will not be described again.

[0397] The first sub-window 104 can further include a second window dynamic effect 107, which can be a colored frame with a flowing light effect surrounding the first panel 106, or a colored frame with a flowing light effect surrounding the first interaction area 98, or other types of dynamic effects, and the embodiments of the present application do not limit the type of the second window dynamic effect 107.

[0398] It should be noted here that in the process of switching the first sub-window 104 to the first window 58 in other states, the second window dynamic effect 107 can switch between different dynamic effects contained therein, for example, from a colored frame with a flowing light effect surrounding the first interaction area 98 to a colored frame with a flowing light effect surrounding the first panel 106.

[0399] The first sub-window 104 can have a second height H2, which can be greater than the first height H1, and for example, can account for 40%-50% of the total height of the display screen, and the embodiments of the present application do not limit the specific height of the second height H2. Based on the second height H2, the first sub-window 104 can accommodate the interaction content between the virtual assistant and the user through the first panel 106.

[0400] In the first sub-window 104, the navigation bar 61 is displayed in the top area of the first panel 106, and the first control 59 can be displayed in the first panel 106 and in the area adjacent to the outer periphery of the first interaction area 98.

[0401] It should be noted that the first control 59 can have different display forms for the first window 58 in different window states, for example, the first control 59 in the first sub-window 104 can be provided with a dynamic effect to improve the visual experience of the user. The dynamic effect can be, for example, a blue background. The display form of the first control 59 is not limited in the embodiments of the present application.

[0402] As shown in C of FIG. 21, the second sub-window 105 includes the first interaction area 98 and a second panel 108 displayed on a part of the display screen of the electronic device 900.

[0403] The second sub-window 105 can further include a third window dynamic effect 109, which can be a colored frame with a flowing light effect surrounding the second panel 108, or a colored frame with a flowing light effect surrounding the first interaction area 98, or other types of dynamic effects. The type of the third window dynamic effect 109 is not limited in the embodiments of the present application.

[0404] It should be noted that the third window dynamic effect 109 can switch between different dynamic effects contained therein during the switching of the second sub-window 105 to the first window 58 in other states, for example, from the colored frame with the flowing light effect surrounding the first interaction area 98 to the colored frame with the flowing light effect surrounding the second panel 108.

[0405] The second sub-window 105 can have a third height H3, which can be greater than the second height H2. The third height H3 can account for, for example, 60%-80% of the total height of the display screen. The specific height of the third height H3 is not limited in the embodiments of the present application. Based on the third height H3, the second sub-window 105 can accommodate more interaction content between the virtual assistant and the user through the second panel 108.

[0406] As shown in D of FIG. 21, the third state window 97 can be displayed in the form of a third panel 110 or in the form of an in-application page corresponding to the virtual assistant.

[0407] A page is usually the entire area of a user interface. A panel is usually a specific area in a user interface, which can be floating or fixed on a page. The specific display form of the third state window 97 is not limited in the embodiments of the present application. Based on the third state window 97, the user can operate in the application corresponding to the virtual assistant.

[0408] Regardless of whether the third state window 97 is displayed in the form of the third panel 110 or in the form of the in-application page corresponding to the virtual assistant, the third state window 97 can cover the full screen area.

[0409] The third state window 97 is exemplarily illustrated in the form of the third panel 110 displayed by the third state window 97.

[0410] The third panel 110 is displayed in a full-screen area corresponding to at least one display screen. The third panel 110 can be used not only to accommodate the interaction content between the virtual assistant and the user, but also to provide more functions. Specifically, the third panel 110 can include multiple sub-panels, such as a virtual assistant panel, an agent store panel, and a recent conversation panel.

[0411] The virtual assistant panel is a panel for the user to interact with the virtual assistant, and is the default sub-panel in the third panel 110. The virtual assistant panel can display the first interaction area 98.

[0412] The agent store panel can provide multiple agent applications. Different agent applications can be suitable for different use scenarios, such as customer service, education, entertainment, medical treatment, and the like. The user can select a corresponding agent application from the agent store panel and apply it to the virtual assistant to meet the user's use requirements.

[0413] The recent conversation panel can store historical conversation content.

[0414] The third panel 110 can be provided with a switching button 111, which can include a virtual assistant button, an agent store button, and a recent conversation button. The virtual assistant button corresponds to the virtual assistant panel, the agent store button corresponds to the agent store panel, and the recent conversation button corresponds to the recent conversation panel.

[0415] The user can perform a click operation on the switching button 111. In response to the click operation of the user, the electronic device 900 can display the sub-panel corresponding to the switching button 111 to perform a corresponding function on the sub-panel.

[0416] It should be noted that in the case where the third panel 110 is the agent store panel or the recent conversation panel, no interaction operation is required on these panels, and therefore the electronic device 900 does not display the first interaction area 98 on the agent store panel and the recent conversation panel, but displays the navigation bar 61. That is, on these panels, the navigation bar 61 and the first interaction area 98 cannot be displayed simultaneously. In fact, in any embodiment of the present application, the navigation bar 61 cannot be displayed simultaneously with the first interaction area 98.

[0417] The third state window 97 in the following embodiments of the present application refers to a window displaying the virtual assistant panel.

[0418] The third state window 97 can further include a fourth window dynamic effect 112. The fourth window dynamic effect 112 can be a color frame with a flowing light effect surrounding the third panel 110, can be a color frame with a flowing light effect surrounding the first interaction area 98, or can be another type of dynamic effect. The embodiments of the present application do not limit the type of the fourth window dynamic effect 112.

[0419] It should be noted that during the switching of the third state window 97 to the first window 58 in another state, the fourth window dynamic effect 112 can switch between different dynamic effects contained therein, for example, from a color frame with a flowing light effect surrounding the first interaction area 98 to a color frame with a flowing light effect surrounding the third panel 110.

[0420] The third state window 97 can have a fourth height H4, which can account for 90%-100% of the total height of the display screen. The embodiments of the present application do not limit the specific height of the fourth height H4. Based on the fourth height H4, the third state window 97 can accommodate more interaction content between the virtual assistant and the user through the third panel 110.

[0421] In order to conform to the operation habits of the user, the navigation bar 61 can not be displayed in the third state window 97, so that the third state window 97 is closer to the display mode of the page.

[0422] In the third state window 97, the first control 59 can be displayed in the virtual assistant panel and in the outer peripheral adjacent area of the first interaction area 98.

[0423] Based on different ways in which the electronic device 900 wakes up the virtual assistant, the first window 58 has different input states.

[0424] FIG. 22 is a schematic diagram of the input states of the first window according to an embodiment of the present application.

[0425] The first state window 60 is exemplarily described for different input states.

[0426] As shown in A of FIG. 22, in a case where the user adopts the first wake-up manner, the second wake-up manner, the fourth wake-up manner, and the fifth wake-up manner (i.e., the manner of waking up the virtual assistant based on the long-pressing of the navigation bar 61, the long-pressing of the home screen button 89, or the drag operation) in the foregoing embodiments, the electronic device 900 displays the first state window 60 with a first input state in response to the wake-up operation of the user. The first input state is a state in which no voice input is currently received.

[0427] In the first input state, the user cannot directly make a voice input to the electronic device 900, but needs to click the voice input button 102 in the first state window 60 to switch to the second input state and then make a voice input. It should be noted that the voice input button 102 in the embodiments of the present application can be set to support a click operation or a long press operation, and the specific setting can be based on the user's usage habits, and the embodiments of the present application do not limit this.

[0428] As shown in B in FIG. 22, in the case where the user adopts the third wake-up manner (i.e., the manner of waking up the virtual assistant based on long pressing the power key 52) in the foregoing embodiments, in response to the wake-up operation of the user, the electronic device 900 displays the first state window 60 with the second input state, which is the state of currently receiving a voice input.

[0429] In the second input state, the user directly makes a voice input to the electronic device 900.

[0430] It should be noted that the first window dynamic effect 103 corresponding to the first state window 60 can be different in different input states. For example, in the first input state, the first window dynamic effect 103 can be a colored frame surrounding the first interaction area 98, in the second input state, the first window dynamic effect 103 can be a halo dynamic effect or a flowing light spot dynamic effect, and in the second input state, the voice input button 102 can be displayed in a highlighted state, and the first assistant icon 100 can display a tilt dynamic effect. The embodiments of the present application do not limit the dynamic effect that the first assistant icon 100 can display.

[0431] The embodiments of the present application are further described with the user waking up the virtual assistant in the first wake-up manner.

[0432] Further as shown in FIG. 14, in the first wake-up manner, the user performs a long press operation on the navigation bar 61 on the first page 87, and in response to the long press operation of the user on the navigation bar 61, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first state window 60 corresponding to the virtual assistant and at least one first control 59 adjacent to the first state window 60, and the first control 59 includes a service recommendation matched with the first page 87.

[0433] The first control 59 is a carrier for inferring a user's scene intention and providing services based on screen understanding, which can provide dynamic service recommendations according to content understanding.

[0434] The first control 59 can provide service recommendations supported by the virtual assistant and service recommendations supported by third-party services. The service recommendations supported by the virtual assistant can be, for example, service recommendations for object recognition, and the service recommendations supported by the third-party services can be service recommendations for jumping to other applications to implement corresponding functions, such as service recommendations for jumping to a note application to save notes. The specific types of service recommendations supported by the first control 59 are not limited in this embodiment of the present application.

[0435] The following exemplary describes some service recommendations supported by the first control 59.

[0436] In a case where the first page 87 includes text content and the text content includes an address, if the address includes at least one of an entity extraction address, a store name, and a scenic spot name, the first control 59 can provide service recommendations for navigation, taking a car, viewing the address, viewing the distance, and collecting.

[0437] For example, the recommended content of the service recommendations includes “navigate to here”, “take a car to here”, “view the address”, “the distance is far”, and “collect”, and the types and the content of the service recommendations are not limited in this embodiment of the present application.

[0438] In a case where the first page 87 includes text content and the text content includes a non-native language, the first control 59 can provide service recommendations for translation.

[0439] For example, the recommended content of the service recommendations includes “translation”.

[0440] In a case where the first page 87 includes picture content and the picture content includes a form interface, if the electronic device 900 detects that the picture is a form interface and the related theme conforms to the business logic of the form, the first control 59 can provide service recommendations for filling the form.

[0441] For example, the recommended content of the service recommendations includes “fill the form”.

[0442] In a case where the first page 87 includes picture content and the picture content includes a chat interface, if the electronic device 900 determines that the picture is a chat record based on label recognition, the first control 59 can provide service recommendations for privacy coding.

[0443] For example, the recommended content of the service recommendations includes “privacy coding”.

[0444] In a case where the first page 87 includes picture content and the picture content includes a product, if the electronic device 900 determines that the picture is a product picture based on label recognition, the first control 59 can provide service recommendations for shopping and collecting.

[0445] For example, the recommended content of the service recommendation includes "shopping" and "collection".

[0446] In a case where the first page 87 includes picture content and the picture content includes a person, if the electronic device 900 detects multiple persons, the first control 59 can provide a service recommendation for automatic removal.

[0447] For example, the recommended content of the service recommendation includes "remove passerby".

[0448] In a case where the first page 87 includes picture content and the picture content includes a person, if the electronic device 900 detects a person face, the first control 59 can provide a service recommendation for automatic beautification.

[0449] For example, the recommended content of the service recommendation includes "one-click skin beautification".

[0450] In a case where the first page 87 includes picture content and the picture content includes a person, if the electronic device 900 detects a person face and the area of the person face is greater than 100 pixels x 100 pixels, the first control 59 can provide a service recommendation for face replacement detection.

[0451] For example, the recommended content of the service recommendation includes "AI face replacement detection".

[0452] In a case where the first page 87 includes picture content and the picture content includes a person, if the electronic device 900 detects a person face and the person face is a front face, the first control 59 can provide a service recommendation for generating an ID photo.

[0453] For example, the recommended content of the service recommendation includes "generate an ID photo".

[0454] In a case where the first page 87 includes picture content and the picture content is a photo taken by a camera, the first control 59 can provide a service recommendation for adding a time and location sticker.

[0455] For example, the recommended content of the service recommendation includes "add a time and location sticker".

[0456] In a case where the first page 87 includes picture content and the picture content includes a subject, the first control 59 can provide a service recommendation for cutout, card information extraction.

[0457] For example, the recommended content of the service recommendation includes "automatic cutout", "extract card information to notes", "save card information to personal information", "copy address", and "copy ID number".

[0458] In a case where the first page 87 includes a desktop custom page, the first control 59 can provide a service recommendation for customizing a desktop.

[0459] For example, the recommended content of the service recommendation includes "customized desktop".

[0460] In a case where the first page 87 includes a video playing page or an audio playing page, the first control 59 can provide a service recommendation for translation.

[0461] For example, the recommended content of the service recommendation includes "AI caption".

[0462] The first control 59 can have different display forms based on different service recommendations.

[0463] FIG. 23 is a schematic diagram of a display manner of the first control according to an embodiment of the present application.

[0464] As shown in A of FIG. 23, the first control 59 can be displayed based on a pure text style.

[0465] As shown in B of FIG. 23, the first control 59 can be displayed based on a star mark and a text style. In this way, the recommendation intention of the first control 59 can be enhanced.

[0466] As shown in C of FIG. 23, the first control 59 can be displayed based on an entity button 113 and a text style. In this way, based on the entity button 113, the electronic device 900 can jump to other services, such as a positioning service, and based on the entity button 113, the electronic device 900 can also implement related functions such as saving and printing. The type of functions that can be implemented by the entity button 113 is not limited in the present application.

[0467] In addition, a plurality of entity buttons 113 can be provided in the same first control 59, so that the same first control 59 implements a plurality of different functions. The number of functions that can be supported by the first control 59 can be, for example, 4, and the number of functions that can be supported by the first control 59 is not limited in the present application.

[0468] As shown in D of FIG. 23, the first control 59 can be displayed based on an application icon 114 and a text style. In this way, based on the first control 59, the electronic device 900 can jump to an application program corresponding to the application icon 114.

[0469] The display form of the first control 59 is not limited in the present application.

[0470] Further as shown in FIG. 14, the outer periphery adjacent area of the first status window 60 can be provided with a plurality of first controls 59, for example, provided with 8 first controls 59, wherein a part of the first controls 59 can be displayed in the outer periphery adjacent area of the first status window 60, for example, 2 first controls 59 are displayed in the outer periphery adjacent area of the first status window 60, and another part of the first controls 59 can be hidden in the outer periphery adjacent area of the first status window 60, for example, 6 first controls 59 are hidden in the outer periphery adjacent area of the first status window 60.

[0471] The user can perform a sliding operation in the outer periphery adjacent area of the first status window 60, and in response to the sliding operation of the user, the electronic device 900 can alternately display the first controls 59.

[0472] The user can perform a clicking operation on the first controls 59, and in response to the clicking operation of the user on the first controls 59, the electronic device 900 can execute a function corresponding to the service recommendation in the first controls 59.

[0473] That is, when the electronic device 900 wakes up the virtual assistant on the first page 87, and displays the first status window 60 corresponding to the virtual assistant, and at least one first control 59 adjacent to the first status window 60, the user can perform an input operation on the first status window 60, or perform a clicking operation on the first controls 59 to interact with the virtual assistant.

[0474] Further, based on other wake-up methods, after the electronic device 900 wakes up the virtual assistant on the first page 87, the electronic device 900 can also have other operation forms.

[0475] The fourth wake-up method is further described in the embodiments of the present application.

[0476] FIG. 24 is a schematic diagram of a display mode of a second control according to an embodiment of the present application.

[0477] As shown in FIGS. 18, 19 and 24, the gesture navigation method of the electronic device 900 is exemplarily described.

[0478] The user can wake up the virtual assistant on the first page 87 by dragging the first target content 90 of the first page 87 to the navigation bar 61.

[0479] Wherein, when the first moving area 92 reaches the second transition window 96, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first status window 60 corresponding to the virtual assistant. The scenario in which the user further drags the first moving area 92 to the first status window 60 is shown in FIG. 19.

[0480] In fact, the user can not need to use the first target content 90 in the first moving area 92 in real time. Then, in the embodiment of the present application, the user can not further drag the first moving area 92 to the first status window 60, but drag the first moving area 92 to an adjacent area of the first status window 60, so that, in response to the dragging of the first moving area 92 to the adjacent area of the first status window 60, the electronic device 900 generates a second control 115 in the adjacent area of the first status window 60, and the second control 115 is used to cache the first target content 90 for subsequent use. Wherein, the second control 115 can be arranged side by side with the first control 59.

[0481] It should be noted here that the first control 59 can not only provide service recommendations for the content in the first page 87, but also provide service recommendations for the content in the second control 115.

[0482] It should be further noted that the embodiment shows the way of generating the second control 115 based on the dragging operation in the case where the virtual assistant is not woken up. In fact, the embodiment of the present application can also generate the second control 115 based on the dragging operation in the case where the virtual assistant is woken up. The two ways can be referred to each other, and the present application will not be described here. Wherein, in the case where the virtual assistant is woken up, the way of generating the second control 115 based on the dragging operation realizes the display of the virtual assistant and the first page 87.

[0483] The user can perform a click operation on the second control 115 when the first target content 90 is needed, and in response to the click operation of the user on the second control 115, the electronic device 900 can take the second control 115 as the input of the second status window 62.

[0484] FIG. 25 is a schematic diagram of an updating process of the first control according to an embodiment of the present application.

[0485] As shown in FIG. 25, in an implementation, the electronic device 900 can update the first control 59 based on the first target content 90 in the second control 115, and obtain an updated control 116, wherein the updated control 116 includes service recommendations matched with the first target content 90.

[0486] For example, the service recommendations provided by the first control 59 in the initial state for the first page 87 are “polish the article” and “summarize the abstract”. If the first target content 90 is a text content in a non-native language, the first control 59 is updated to the updated control 116, and the service recommendations provided by the second control 115 for the updated control 116 are, for example, “translation”.

[0487] That is, the electronic device 900 can wake up the virtual assistant on the first page 87, and display the first status window 60 corresponding to the virtual assistant, and at least one update control 116 adjacent to the first status window 60, and the second control 115 adjacent to the first control 59.

[0488] In this way, the user can perform an input operation on the first status window 60, a click operation on the update control 116, and a click operation on the second control 115 to interact with the virtual assistant.

[0489] The present application will be further described with the user waking up the virtual assistant in the first way.

[0490] In the first way, the user can perform an input operation on the first status window 60, or a click operation on the first control 59 to interact with the virtual assistant.

[0491] FIG. 26 is a schematic diagram of a scenario in which the user performs an input operation on the first status window according to an embodiment of the present application.

[0492] As shown in FIG. 26, when the first status window 60 is displayed, the user can perform a click operation on the input field 99. In response to the click operation on the input field 99, the electronic device 900 displays the input method panel 117, which covers the first status window 60 so that the first status window 60 is not visible.

[0493] After the user completes the input operation, the user can perform an exit operation on the input method panel 117. The exit operation can be a gesture operation or a click operation on the exit button 118. In response to the operation of exiting the input method panel 117, the electronic device 900 hides the input method panel 117, displays the second status window 62, and displays the interaction content corresponding to the input operation in the second status window 62.

[0494] It should be further noted that the input method panel 117 is actually a system-level pop-up control. Similar system-level pop-up controls include a slidable panel control and the like. The display priority of such controls can be higher than that of the first status window 60. After exiting such controls, the electronic device 900 can display the first status window 60 again. The display priority of the first window 58 in other states can be the same as that of the first status window 60. The present application will not be described in detail.

[0495] It should be noted that the first state window 60 has an automatic exit mechanism (which will be described in subsequent embodiments of the present application). During the process of displaying the input method panel 117 by the electronic device 900, the automatic exit mechanism of the first state window 60 is suspended. The present embodiment is exemplarily described by taking the first state window 60 as an example, and the first window 58 in other states is the same. The present application will not be described here.

[0496] In the above embodiment, after the user completes the input operation, the electronic device 900 switches from displaying the first state window 60 to displaying the second state window 62. In the present embodiment, there are various switching modes between the first window 58 in different states in different scenarios, and the switching mode of the first window 58 will be described below.

[0497] Specifically, during the process of the user interacting with the virtual assistant, the electronic device 900 can switch the window state of the first window 58 based on automatic switching or based on gesture operation, so as to meet different use requirements of the user.

[0498] FIG. 27 is a first scenario diagram of the first window switching the window state according to an embodiment of the present application.

[0499] As shown in FIG. 27, the user can perform an input operation on the first state window 60, and in response to the input operation on the first state window 60, the electronic device 900 displays the first sub-window 104 and displays the interaction content corresponding to the input operation in the first panel 106.

[0500] It should be noted that the input operation can be an operation based on the input method panel 117 or an operation based on the voice input button 102. The specific operation form of the input operation is not limited in the present embodiment.

[0501] Based on different instructions corresponding to the input operation, the electronic device 900 can provide a response content, automatically execute a corresponding operation corresponding to the response content, or jump to other application programs based on the response content, so as to enable the user to use other application programs quickly.

[0502] Exemplarily, the input operation based on the voice input button 102 is exemplarily described. For example, the user inputs a voice instruction of “turn on Bluetooth” to the first state window 60 based on the voice input button 102, and the electronic device 900 displays the first sub-window 104 and displays the text content converted from the voice instruction in the first sub-window 104. Based on the virtual assistant, the electronic device 900 can display a response content in the first sub-window 104, and the response content can be, for example, “Bluetooth has been turned on for you”. Moreover, the electronic device 900 can automatically execute a corresponding operation corresponding to the response content.

[0503] To ensure that the automatically performed operation meets the actual needs of the user, the electronic device 900 can display a response control associated with the response content in the first sub-window 104, which can be, for example, the Bluetooth control 119. The Bluetooth control 119 can include a Bluetooth switch 120, which is displayed in an on state to indicate that the electronic device 900 has currently automatically performed the operation of turning on Bluetooth. If the operation automatically performed by the electronic device 900 does not meet the needs of the user, the user can manually turn off the response control.

[0504] FIG. 28 is a schematic diagram of a second scenario of window state switching of the first window according to an embodiment of the present application.

[0505] As shown in FIG. 28, an example is illustratively described based on an input operation of the input method panel 117. For example, the user inputs text content of “introduce artificial intelligence” based on the input method panel 117, and the electronic device 900 displays the first sub-window 104 and displays the response content corresponding to the text content in the first sub-window 104.

[0506] Further, if the user inputs a voice instruction of “store the current content to the note application” based on the first sub-window 104 and the voice input button 102, the electronic device 900 can jump to the note application and display the response content corresponding to the text content on a page in the note application. In the embodiment of the present application, the electronic device 900 supports jumping to any application by the virtual assistant, and the type of the application that can be jumped to is not limited in the embodiment of the present application.

[0507] FIG. 29 is a schematic diagram of a third scenario of window state switching of the first window according to an embodiment of the present application.

[0508] As shown in FIG. 29, if the window size of the first sub-window 104 cannot accommodate the response content, the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105 and displays the response content in the second panel 108.

[0509] Specifically, the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105 can be triggered in a case where the electronic device 900 detects that the display area of the current interaction content is greater than a preset first area threshold. The first area threshold can be, for example, equal to 80% of the total window area of the first sub-window. The specific value of the first area threshold is not limited in the embodiment of the present application.

[0510] That is, the electronic device 900 can automatically trigger the switching from displaying the first sub-window 104 to displaying the second sub-window 105 based on the interaction content, so that the second sub-window 105 provides a larger display area for the interaction content, thereby improving the visual experience of the user.

[0511] It should be noted that after the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105, subsequent interaction content can be displayed in the second sub-window 105, and the electronic device 900 no longer automatically triggers switching from displaying the second sub-window 105 to displaying the first sub-window 104.

[0512] The electronic device 900 can also switch the window state of the first window 58 based on the manner of the gesture operation.

[0513] FIG. 30 is a fourth scenario diagram of switching the window state of the first window according to an embodiment of the present application.

[0514] As shown in FIG. 30, when the first sub-window 104 is displayed, if the user considers that the display area of the first sub-window 104 is too small, the user can perform an upward sliding operation on the navigation bar 61 in the first sub-window 104. In response to the upward sliding operation on the navigation bar 61 in the first sub-window 104, the electronic device 900 can display a third state window 97.

[0515] The interaction content in the first sub-window 104 can be synchronously displayed in the third state window 97. In this way, the third state window 97 provides a larger display area for the interaction content, and improves the visual experience of the user.

[0516] FIG. 31 is a fifth scenario diagram of switching the window state of the first window according to an embodiment of the present application.

[0517] As shown in FIG. 31, the electronic device 900 can automatically trigger switching from displaying the first sub-window 104 to displaying the second sub-window 105 based on the interaction content.

[0518] If the user considers that the display area of the second sub-window 105 is too small, the user can perform an upward sliding operation on the navigation bar 61 in the second sub-window 105. In response to the upward sliding operation on the navigation bar 61 in the second sub-window 105, the electronic device 900 can display the third state window 97.

[0519] The interaction content in the second sub-window 105 can be synchronously displayed in the third state window 97. In this way, the third state window 97 provides a larger display area for the interaction content, and improves the visual experience of the user.

[0520] It should be noted that the embodiments of the present application only exemplarily describe the manner in which the first sub-window 104 is switched to the third state window 97, and the manner in which the second sub-window 105 is switched to the third state window 97. In fact, in the embodiments of the present application, when the first state window 60 is displayed, in response to the upward sliding operation on the navigation bar 61 in the first state window 60, the electronic device 900 can display at least one of the first sub-window 104, the second sub-window 105, and the third state window 97. When the first sub-window 104 is displayed, in response to the upward sliding operation on the navigation bar 61 in the first sub-window 104, the electronic device 900 can display at least one of the second sub-window 105 and the third state window 97.

[0521] The window actually displayed by the electronic device 900 can be determined based on the position at which the user performs the upward sliding operation. The electronic device 900 can set the switching manner between the first windows 58 in any two display states to support gesture switching, or can set the switching manner between the first windows 58 in any two display states to not support gesture switching, which is not limited in the embodiments of the present application.

[0522] FIG. 32 is a schematic diagram of a sixth scenario in which the first window is switched in a window state according to an embodiment of the present application.

[0523] As shown in FIG. 32, when the electronic device 900 displays the third state window 97, if the user wants to simultaneously operate the first page 87, the user can switch the third state window 97 to the first state window 60.

[0524] The user can perform a downward sliding operation on the top region of the display screen of the electronic device 900, and in response to the downward sliding operation on the top region of the display screen of the electronic device 900, the electronic device 900 is switched from displaying the third state window 97 to displaying the first state window 60.

[0525] It should be noted that the embodiments of the present application only exemplarily describe the manner in which the third state window 97 is switched to the first state window 60. In fact, in the embodiments of the present application, when the third state window 97 is displayed, in response to the downward sliding operation on the top region of the display screen of the electronic device 900, the electronic device 900 can display at least one of the first state window 60, the first sub-window 104, and the second sub-window 105. When the second sub-window 105 is displayed, in response to the downward sliding operation on the navigation bar 61 in the second sub-window 105, the electronic device 900 can display at least one of the first state window 60 and the first sub-window 104. When the first sub-window 104 is displayed, in response to the downward sliding operation on the navigation bar 61 in the first sub-window 104, the electronic device 900 can display the first state window 60.

[0526] The window displayed by the electronic device 900 can be determined based on the position of the downward sliding operation performed by the user. The electronic device 900 can set the switching mode between the first window 58 of any two display states to support gesture switching, or can set the switching mode to not support gesture switching, and the embodiments of the present application do not limit this. For example, if the electronic device 900 sets the switching between the third state window 97 and the second sub-window 105 to not support the downward sliding operation, the electronic device 900 cannot switch from the third state window 97 to the second sub-window 105 in response to the downward sliding operation.

[0527] FIG. 33 is a seventh scenario diagram of window state switching of the first window according to an embodiment of the present application.

[0528] As shown in FIG. 33, in a case where the electronic device 900 displays the third state window 97, if the user wants to end the virtual assistant session, the user can perform a downward sliding operation on the navigation bar 61 in the third state window 97. In response to the downward sliding operation on the navigation bar 61 in the third state window 97, the electronic device 900 closes the third state window 97 and displays the navigation bar 61 at the bottom of the display screen.

[0529] Correspondingly, in a case where the electronic device 900 displays the second state window 62, if the user wants to end the virtual assistant session, the user can perform a downward sliding operation on the navigation bar 61 in the second state window 62. In response to the downward sliding operation on the navigation bar 61 in the second state window 62, the electronic device 900 closes the second state window 62 and displays the navigation bar 61 at the bottom of the display screen. This specific implementation mode can be referred to the third state window 97, and the present application does not show the diagram.

[0530] It should be noted that in the second state window 62 and the third state window 97, if there is interaction content, the second state window 62 and the third state window 97 can cache the interaction content to generate historical content, which is displayed when the virtual assistant is subsequently awakened again.

[0531] Correspondingly, in a case where the electronic device 900 displays the first state window 60, if the user wants to end the virtual assistant session, the user can perform a downward sliding operation on the navigation bar 61 in the first state window 60. In response to the downward sliding operation on the navigation bar 61 in the first state window 60, the electronic device 900 closes the first state window 60 and displays the navigation bar 61 at the bottom of the display screen. This specific implementation mode can be referred to the third state window 97, and the present application does not show the diagram.

[0532] Further, in the process of interaction between the user and the virtual assistant, the electronic device 900 can also move the first window 58 based on the gesture operation mode to meet different use requirements of the user.

[0533] FIG. 34 is a first scene diagram of moving a first window of an electronic device according to an embodiment of the present application.

[0534] As shown in FIG. 34, in a case where the first state window 60 is displayed, the user can perform a long-pressing and dragging operation on the navigation bar 61 in the first state window 60. In response to the long-pressing and dragging operation on the navigation bar 61 in the first state window 60, the electronic device 900 displays the first state window 60 as a corresponding moving state window 121, and the moving state window 121 includes the first interaction area 98 and a fourth panel 122 for accommodating the first control 59.

[0535] The user can drag the moving state window 121 to a target position 123. In response to the operation of dragging the moving state window 121 to the target position 123, the electronic device 900 displays the moving state window 121 at the target position 123.

[0536] The display mode of the moving state window 121 is more suitable for display in the central area of the display screen, which can improve the visual experience and operation experience of the user. When the user performs the long-pressing and dragging operation on the navigation bar 61 in the first state window 60, the navigation bar 61 can change in dynamic effect, for example, change in color, such as from gray to blue. The present application does not limit the specific manner in which the navigation bar 61 can change in dynamic effect.

[0537] FIG. 35 is a second scene diagram of moving a first window of an electronic device according to an embodiment of the present application.

[0538] As shown in FIG. 35, in a case where the first sub-window 104 is displayed, the user can perform a long-pressing and dragging operation on the navigation bar 61 in the first sub-window 104. In response to the operation of long-pressing and dragging the navigation bar 61 in the first sub-window 104 to the target position 123, the electronic device 900 displays the first sub-window 104 at the target position 123.

[0539] When the user performs the long-pressing and dragging operation on the navigation bar 61 in the first sub-window 104, the navigation bar 61 can change in dynamic effect, for example, change in color, such as from gray to blue.

[0540] The first sub-window 104 has a suitable window size, so that no morphological change is required for the first sub-window 104 in the process of moving the first sub-window 104.

[0541] FIG. 36 is a third scene diagram of moving a first window of an electronic device according to an embodiment of the present application.

[0542] As shown in FIG. 36, in the case of displaying the second sub-window 105, the user can perform a long-pressing and dragging operation on the navigation bar 61 in the second sub-window 105, and in response to the long-pressing and dragging operation on the navigation bar 61 in the second sub-window 105, the electronic device 900 reduces the height of the second panel 108 to the height of the first panel 106, so as to transform the second sub-window 105 into the first sub-window 104. The user can drag the first sub-window 104 to the target position 123, and in response to the operation of dragging the first sub-window 104 to the target position 123, the electronic device 900 displays the first sub-window 104 at the target position 123.

[0543] In the operation of the user long-pressing and dragging the navigation bar 61 in the second sub-window 105, the navigation bar 61 can change in animation, for example, change in color, such as from gray to blue.

[0544] The window size of the second sub-window 105 occupies a large proportion in the display screen, and therefore, in the process of moving the second sub-window 105, the morphological change of the second sub-window 105 can improve the visual experience of the user.

[0545] It should be noted that the above-mentioned first to third scenarios of the electronic device moving the first window are implemented in the same way for the foldable electronic device and the non-foldable electronic device, and the embodiments of the present application are exemplarily described by taking the non-foldable electronic device. The implementation of the foldable electronic device can refer to the non-foldable electronic device.

[0546] FIG. 37 is a schematic diagram of a fourth scenario of the electronic device moving the first window according to an embodiment of the present application.

[0547] As shown in FIG. 37, in the case of displaying the third state window 97, the user can perform a long-pressing and dragging operation on the top area of the display screen, and in response to the long-pressing and dragging operation on the top area of the display screen, the electronic device 900 can display the third state window 97 as the second state window 62. The user can drag the second state window 62 to the target position 123, and in response to the operation of dragging the second state window 62 to the target position 123, the electronic device 900 displays the second state window 62 at the target position 123.

[0548] Among them, in the non-foldable electronic device, the electronic device 900 can display the third state window 97 as the first sub-window 104, and in the foldable electronic device, the electronic device 900 can display the third state window 97 as the second sub-window 105. This display mode can be determined based on the screen area difference between the non-foldable electronic device and the foldable electronic device. In fact, in the foldable electronic device, the electronic device 900 can also display the third state window 97 as the first sub-window 104. The specific display form of the second state window 62 is not limited in this embodiment of the application. This embodiment of the application is only exemplarily described with the foldable electronic device.

[0549] FIG. 38 is a fifth scenario schematic diagram of the electronic device moving the first window provided in an embodiment of the application.

[0550] As shown in FIG. 38, in the case that the electronic device 900 is a foldable electronic device, the foldable electronic device is in an unfolded state, and the third state window 97 is displayed in the full-screen area corresponding to the entire display screen, the user can perform a long press on the top area of the display screen and drag along the top edge. In response to the operation of long pressing on the top area of the display screen and dragging along the top edge, based on the split-screen mode, the electronic device 900 can display the third state window 97 in the full-screen area corresponding to a part of the display screen, that is, in the split screen on one side, and display the first page 87 in the full-screen area corresponding to another part of the display screen, that is, in the split screen on the other side.

[0551] The above describes the input operation of the user on the first state window 60 in the first wake-up mode, and the window state switching operation and the moving window operation performed after the input operation.

[0552] Further, in the first wake-up mode, the user can also perform a click operation on the first control 59. In response to the click operation of the user on the first control 59, the electronic device 900 can execute the function corresponding to the service recommendation in the first control 59.

[0553] FIG. 39 is a first scenario schematic diagram in which the user interacts with the virtual assistant based on the first control provided in an embodiment of the application.

[0554] As shown in FIG. 39, in the case that the service recommendation in the first control 59 can provide a text service (for example, a summary abstract), the user performs a click operation on the first control 59. In response to the click operation on the first control 59, the electronic device 900 displays the first sub-window 104 and displays the text service content corresponding to the service recommendation in the first sub-window 104.

[0555] If the window size of the first sub-window 104 cannot accommodate the text service content, the electronic device 900 switches from displaying the first sub-window 104 to displaying the second sub-window 105, and displays the text service content in the second panel 108. The manner in which the first sub-window 104 switches to display the second sub-window 105 in this embodiment can refer to the foregoing embodiments, and will not be described herein. The embodiments of this application are only exemplarily described by taking displaying the first sub-window 104 as an example.

[0556] It should be noted that the service recommendation in the first control 59 is recommended according to the current content of the first page 87. If the current content of the first page 87 is updated, the first control 59 is updated accordingly.

[0557] For example, when the first page 87 is a mail page, if the current content of the first page 87 is the mail body, the first control 59 can provide a service recommendation according to the mail body, for example, "polish the article" and "summarize the abstract". If the current content of the first page 87 is the mail address, the first control 59 can provide a service recommendation according to the mail address, for example, "navigation" and "make a call".

[0558] If there is no selected area in the first page 87, the first control 59 can provide a service recommendation according to the overall content currently displayed by the first page 87. If there is a selected area in the first page 87, for example, the first selected area 91 in the foregoing embodiments, the first control 59 can provide a service recommendation according to the selected area in the first page 87.

[0559] It should be further noted that the embodiments of this application not only support providing a service recommendation for a selected area in the first page 87 through the first control 59 when the virtual assistant is woken up, but also support providing a service recommendation for a selected area in the first page 87 through the first control 59 when the virtual assistant is not woken up. When the virtual assistant is not woken up, the electronic device 900 only displays the navigation bar 61 at the bottom of the first page 87, and does not display the first window 58 in any window form. Then, the electronic device 900 can display the first control 59 in the adjacent area of the navigation bar 61.

[0560] Further, in order to be closer to the use intention of the user, the electronic device 900 can display the first control 59 in the adjacent area of the navigation bar 61 when there is a selected area in the first page 87 and the user further performs a long press on the selected area to call out a copy button. The specific triggering form of displaying the first control 59 is not limited in the embodiments of this application.

[0561] In addition, in order to give a strong prompt to the user, the first control 59 can display a dynamic effect, for example, a flowing light dynamic effect. The embodiment of the present application does not limit the type of dynamic effect of the first control 59.

[0562] In fact, in the case of not waking up the virtual assistant, the electronic device 900 displays the first control 59 in the adjacent area of the navigation bar 61, which may not meet the user's needs. Therefore, if the user does not operate the first control 59 within a preset time period, the electronic device 900 hides the first control 59. The preset time period may be, for example, 5s. The user can also perform a downward sliding operation on the first control 59, and in response to the downward sliding operation, the electronic device 900 hides the first control 59.

[0563] FIG. 40 is a schematic diagram of a second scenario in which the user interacts with the virtual assistant based on the first control according to an embodiment of the present application.

[0564] As shown in A in FIG. 40, in the case where the service recommendation in the first control 59 provides a third-party service (for example, shopping), the user performs a click operation on the first control 59, and in response to the click operation on the first control 59, the electronic device 900 jumps to a shopping application, displays a shopping page 124a corresponding to the shopping application, and ends the virtual assistant session.

[0565] As shown in B in FIG. 40, based on a plurality of first controls 59, the electronic device 900 can provide a plurality of third-party services, which can be the same type of service or different types of services. The embodiment of the present application does not limit the type of third-party service provided.

[0566] Taking the case of providing the same type of service by the electronic device 900 as an example, in the case of providing a shopping service, each first control 59 can be used to jump to a different shopping application. In the first control 59, a shopping T icon that can jump to a shopping application T1, a shopping D icon that can jump to a shopping application D1, and a shopping P icon that can jump to a shopping application P1 can be displayed.

[0567] The user can perform a click operation on any first control 59, for example, a click operation on the first control 59 corresponding to the shopping T icon. In response to the click operation on the first control 59, the electronic device 900 jumps to the shopping application T1 and displays a shopping page 124b corresponding to the shopping application T1. In addition, the first state window 60 can also be displayed on the shopping page 124b, and the remaining first controls 59 except the first control 59 that has been clicked can be displayed in the adjacent area of the first state window 60, for example, the first control 59 corresponding to the shopping D icon and the first control 59 corresponding to the shopping P icon. In this way, for the same type of service, the electronic device 900 provides multiple selection methods for the user, which can meet the user's needs.

[0568] Further, within a preset time length, if the user does not click the remaining first controls 59, the electronic device 900 hides the remaining first controls 59, for example, the first controls 59 corresponding to the shopping D icon and the first controls 59 corresponding to the shopping P icon.

[0569] FIG. 41 is a third scenario diagram of user interaction with the virtual assistant based on the first control according to an embodiment of the present application.

[0570] As shown in FIGS. 34 and 41, if the user performs a moving operation on the first state window 60 in the first wake-up mode, the electronic device 900 currently displays a moving state window 121. If the first control 59 in the moving state window 121 provides a third-party service (for example, shopping), when the user performs a click operation on the first control 59, the electronic device 900 jumps to a shopping application and displays a shopping page 124 corresponding to the shopping application, and displays the moving state window 121 in the shopping page 124. That is, unlike the embodiment corresponding to FIG. 40, if the first control 59 corresponding to the moving state window 121 jumps to the shopping application, the electronic device can continue to provide the virtual assistant service based on the moving state window 121, and the virtual assistant session is not ended. The first control 59 can further provide service recommendations to the shopping page 124, meeting the user's use requirements.

[0571] The above describes the way in which the user interacts with the virtual assistant on the first page 87 in the first wake-up mode.

[0572] Further, after the user interacts with the virtual assistant, the user can further interact with the first page 87 to realize the display of the virtual assistant and the first page 87.

[0573] As shown in FIGS. 27 to 38, after the user interacts with the virtual assistant, the first page 87 can currently display any one of the first state window 60, the second state window 62, and the third state window 97.

[0574] It should be noted here that the user interaction with the virtual assistant not only refers to the user waking up the virtual assistant, but also includes the interaction realized by the user performing input, moving, switching, and the like on the first window 58 corresponding to the virtual assistant.

[0575] If the user only interacts with the virtual assistant based on the wake-up operation, the first page 87 can display the first status window 60 based on the user's long press operation on the navigation bar 61, or display the second status window 62 based on the user's drag operation. The first page 87 usually does not directly display the third status window 97, which is usually displayed based on the user's slide gesture operation.

[0576] During the user's interaction with the first page 87 again, the display state of the first status window 60, the second status window 62, and the third status window 97 (herein referred to as the third status window 97 in split-screen mode) does not change, regardless of whether the first status window 60, the second status window 62, or the third status window 97 is currently displayed. That is, the electronic device 900 does not end the session of the virtual assistant.

[0577] The following embodiments of the present application exemplarily illustrate the case where the electronic device 900 displays the first status window 60 on the first page 87 after the user interacts with the virtual assistant. In this case, the user can perform various forms of operations on the first page 87.

[0578] For example, in the case where the first application is a mailbox application, the first page 87 usually displays text content, and the user can perform a swipe-up operation or a swipe-down operation on the first page 87 to browse the text content. The user can also perform an operation to return to a parent page or an operation to enter a child page on the first page 87 to jump between different hierarchical pages in the mailbox application. The user can also perform an operation to jump to another application page and then return to the first page 87, or an operation to jump to another system-level page (such as a notification center or a multi-task page) and then return to the first page 87. The user can also perform a selection operation or a drag operation on the first page to copy or save the text content. The embodiments of the present application do not limit the form of the user's operation on the first page.

[0579] FIG. 42 is a schematic diagram of a first scenario in which the user performs an operation on the first page according to an embodiment of the present application.

[0580] As shown in FIG. 42, in the case where the electronic device 900 displays the first status window 60, the user can perform a swipe-down operation on the first page 87. In response to the user's swipe-down operation on the first page 87, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain a first updated page 125.

[0581] Further, based on the first updated page 125, the electronic device 900 can update the first control 59 to obtain an updated control 126. The updated control 126 includes a service recommendation that matches the first updated page 125.

[0582] Specifically, the electronic device 900 needs a certain update time to update the first control 59, that is, the electronic device 900 can update the first control 59 based on the page content in the first updated page 125 within a first preset time length when the user stops the downward sliding operation, and display the updated control 126 when the first preset time length is reached, where the first preset time length may be, for example, 3 seconds, and the specific value of the first preset time length is not limited in the embodiments of the present application.

[0583] For example, the electronic device displays Chinese content 127 in the first page 87, and displays the first state window 60 and the first control 59, where the service recommendation provided by the first control 59 based on the Chinese content 127 may be, for example, "polish the article" and "summarize the abstract".

[0584] The user performs a downward sliding operation on the first page 87, and in response to the downward sliding operation of the user on the first page 87, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain the first updated page 125. The first updated page 125 may, for example, display English content 128.

[0585] After 3 seconds, the electronic device 900 can update the first control 59 based on the English content 128 in the first updated page 125 to obtain the updated control 126, where the service recommendation provided by the updated control 126 based on the English content 128 may be, for example, "translation".

[0586] In this way, the electronic device 900 can update the corresponding service recommendation in real time according to the updated content in the first page 87. Moreover, the above-mentioned hiding mechanism of the first control 59 can not only improve the visual experience of the user, but also ensure the adaptability of the provided service recommendation.

[0587] FIG. 43 is a second scene diagram of the operation on the first page according to an embodiment of the present application.

[0588] As shown in FIG. 43, when the electronic device 900 displays the first state window 60, the user can perform a selection operation on the first target content 90 in the first page 87, and in response to the selection operation of the user on the first target content 90, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain the first updated page 125.

[0589] Further, based on the first updated page 125, the electronic device 900 can update the first control 59 to obtain the updated control 126. The updated control 126 includes a service recommendation matched with the first updated page 125.

[0590] Specifically, the electronic device 900 needs a certain update time to update the first control 59, that is, the electronic device 900 can update the first control 59 based on the first target content 90 within a first preset time length when the user stops the selection operation, and display the updated control 126 when the first preset time length is reached, where the first preset time length may be, for example, 3 seconds, and the specific value of the first preset time length is not limited in the embodiments of the present application.

[0591] For example, the electronic device displays the Chinese content 127 and the English content 128 in the first page 87, displays the first state window 60, and displays the first control 59, where the service recommendation provided by the first control 59 based on the Chinese content 127 may be, for example, “polish the article” and “summarize the abstract”.

[0592] The user can perform a selection operation on the English content 128, and in response to the selection operation of the user on the English content 128, the electronic device 900 can update the first page 87 and hide the first control 59 to obtain a first updated page 125. The first updated page 125 may be, for example, a page highlighting the English content 128.

[0593] After 3 seconds, the electronic device 900 can update the first control 59 based on the English content 128 in the first updated page 125 to obtain the updated control 126, where the service recommendation provided by the updated control 126 based on the English content 128 may be, for example, “translation”.

[0594] In this way, the electronic device 900 can update the corresponding service recommendation in real time according to the updated content in the first page 87. Moreover, the above-mentioned hiding mechanism of the first control 59 can not only improve the visual experience of the user, but also ensure the adaptability of the provided service recommendation.

[0595] Further, in the embodiments of the present application, the electronic device 900 can set the priority of different operation forms, for example, the priority of the selection operation can be set to be higher than the priority of the sliding operation, so that when the user has selected the content of a certain region in the first page 87, if the user performs a sliding operation again, the electronic device 900 can preferentially update the service recommendation in real time based on the selection operation. The setting manner of the priority of different operation forms is not limited in the embodiments of the present application.

[0596] FIG. 44 is a third scenario diagram of the user operating the first page according to an embodiment of the present application.

[0597] As shown in FIGS. 43 and 44, after the user performs the selecting operation on the first page 87, the electronic device 900 has updated the first control 59 to obtain the updated control 126, and the user further performs the long-pressing and copying operation on the selected area, the electronic device 900 updates the updated control 126 based on the copying operation to obtain the updated control 129. The updated control 129 can provide the service recommendation related to the copying operation.

[0598] Further, as shown in the foregoing embodiments, the service recommendation provided by the updated control 126 based on the selected area may, for example, be "translation". Then, the electronic device 900 updates the updated control 126 based on the copying operation to obtain the updated control 129, wherein the service recommendation provided by the updated control 129 based on the copying operation may, for example, be "save notes".

[0599] That is, the embodiment can set the priority of the copying operation to be greater than the priority of the selecting operation.

[0600] It should be noted that the manner in which the electronic device 900 updates the service recommendation in real time under other operation forms (for example, a downward sliding operation, an operation of returning to a parent page, or an operation of entering a child page) can refer to the foregoing embodiments, and will not be described herein.

[0601] Further, as known from the embodiments corresponding to FIGS. 42 and 43, the electronic device 900 sets a hiding mechanism for the first control 59, and in the embodiment of the present application, the electronic device 900 further sets a hiding mechanism for the first window 58 to improve the visual experience of the user.

[0602] FIG. 45 is a fourth scenario diagram of a user performing an operation on a first page according to an embodiment of the present application.

[0603] As shown in FIG. 45, in a case where the electronic device 900 displays the first state window 60, the user can perform an operation of sliding from the bottom edge to the top edge of the display screen, and in response to the operation of sliding from the bottom edge to the top edge of the display screen, the electronic device 900 displays the multi-task page 130 and hides the first state window 60 and the first control 59. Further, the user can perform an operation of returning from the multi-task page 130 to the first page 87 again, and in response to the operation of returning from the multi-task page 130 to the first page 87 again, the electronic device 900 displays the first page 87.

[0604] It should be noted that the hiding mechanism provided by the electronic device 900 in other operation modes (for example, jumping to other system-level pages such as a notification control center page, a minus one screen page, a right two screen page, and the like, and the type of the system-level page is not limited in the embodiments of the present application) can be referred to the embodiments, and the present application will not be described here.

[0605] In this way, based on the above operation mode, the user can cooperatively use the virtual assistant and the first application.

[0606] Since the user can interact multiple times in the first page 87 and the first state window 60, in order to avoid conflicts of gesture operations in the interaction process, the embodiments of the present application set the related mechanism of focus switching to ensure the accuracy of identifying the user behavior.

[0607] FIG. 46 is a first scene diagram of focus switching provided by the embodiments of the present application.

[0608] As shown in FIG. 46, a first scene in which the user operates the first page is exemplarily illustrated.

[0609] In the case that the user performs a downward sliding operation on the first page 87, in response to the downward sliding operation of the user on the first page 87, the electronic device 900 can set a first focus on the first page 87 to focus on the first page 87, and then control the downward sliding operation to match the first page 87. That is, the current downward sliding operation will not affect the first state window 60.

[0610] In the case that the downward sliding operation matches the first page 87, the electronic device 900 controls the first page 87 to update corresponding to the downward sliding operation to obtain a first updated page 125.

[0611] At this time, the electronic device 900 displays the first state window 60 as a defocused state. In the defocused state, the first state window 60 does not respond to the current gesture operation, and the first window effect 103 corresponding to the defocused state is different from the normal state. For example, the first window effect 103 can be a dynamic effect of morphological reduction and no color frame display. The specific type of the first window effect 103 corresponding to the defocused state is not limited in the embodiments of the present application.

[0612] Correspondingly, in the case that the first page 87 displays the second state window 62, the window effect corresponding to the defocused state is different from the normal state. For example, the window effect can be a dynamic effect of no color frame display. The embodiments of the present application only exemplarily illustrate the defocused state of the first state window 60. FIG. 47 is a second scene diagram of focus switching provided by the embodiments of the present application.

[0613] As shown in FIGS. 46 and 47, the user can perform a click operation on the first status window 60 again. In response to the click operation on the first status window 60, the electronic device 900 moves the first focus to the first status window 60, so that the first status window 60 is displayed from the defocus state to the focus state.

[0614] In the case where the first status window 60 is displayed in the focus state, the electronic device 900 focuses on the first status window 60, and then controls at least one of the up swipe operation, the down swipe operation, and the input operation to be matched with the first status window 60. In the case where the related operation with respect to the first status window 60 is matched with the first status window 60, the electronic device 900 controls the first status window 60 to respond to the related operation.

[0615] In the focus state, the first window effect 103 of the first status window 60 can restore the initial effect.

[0616] Further, in the case where the electronic device 900 is a foldable electronic device and the foldable electronic device is in the unfolded state, since the first page 87 can display more content, the related mechanism of the focus switching can be further set for the first page 87 in the split-screen mode.

[0617] FIG. 48 is a third scenario diagram of focus switching provided by an embodiment of the present application.

[0618] As shown in A of FIG. 48, in the case where the first page 87 is displayed based on the split-screen mode and the first status window 60 is currently displayed, if the first focus stays in the first split-screen page 131 corresponding to the first page 87, the electronic device 900 updates the first control 59 based on the first split-screen page 131 to obtain an updated control 132, and the updated control 132 includes service recommendations matched with the first split-screen page 131.

[0619] For example, the first split-screen page 131 includes Chinese content 127, and the service recommendations provided by the updated control 132 based on the Chinese content 127 can be, for example, “polish the article” and “summarize the abstract”.

[0620] As shown in B of FIG. 48, if the first focus stays in the second split-screen page 133 corresponding to the first page 87, the electronic device 900 updates the first control 59 based on the second split-screen page 133 to obtain the updated control 132, and the updated control 132 includes service recommendations matched with the second split-screen page 133.

[0621] For example, the first split-screen page 131 includes English content 128, and the service recommendations provided by the updated control 132 based on the English content 128 can be, for example, “translation”.

[0622] It should be noted that after the user wakes up the virtual assistant, the user can only perform the clicking operation on the first page 87, and the electronic device 900 sets the first focus on the first page 87, and provides the service recommendation corresponding to the split-screen page where the first focus is located. In fact, the above process is also applicable to the first updated page 125 in the foregoing embodiments. The embodiments of the present application only exemplarily describe the first page 87, and the two pages can be referred to each other, and the embodiments of the present application will not be described here.

[0623] FIG. 49 is a fourth scene diagram of focus switching provided by an embodiment of the present application.

[0624] As shown in FIGS. 48 and 49, on the basis of the scene shown in FIG. 48, the first page 87 can further display the floating window 134. If the first focus stays in the floating window 134, the electronic device 900 updates the first control 59 based on the floating window 134 to obtain the updated control 132, and the updated control 132 includes the service recommendation matched with the floating window 134.

[0625] After the user performs the above interaction process with the virtual assistant, the user can end the session with the virtual assistant.

[0626] FIG. 50 is a first scene diagram of ending the session of the virtual assistant provided by an embodiment of the present application.

[0627] As shown in FIG. 50, a first scene in which the user operates the first page is exemplarily described.

[0628] In the case where the electronic device 900 displays the first state window 60, the user can perform the downward sliding operation on the first page 87. During the interaction between the user and the first page 87, if the user does not use the virtual assistant within a certain time, it indicates that the user currently can not have the intention to use the virtual assistant, and then the electronic device 900 can minimize the first state window 60 to the navigation bar 61 to end the session of the virtual assistant. In addition, since the virtual assistant is still displayed on the first page 87 when the user operates the first page 87, the user can not need to use the virtual assistant again after the interaction with the virtual assistant, and the first state window 60 stays on the first page 87 for a long time, which can easily affect the visual experience of the user.

[0629] Therefore, the embodiments of the present application provide an automatic exit mechanism for the virtual assistant.

[0630] Specifically, in the case where the preset time length is reached after waking up the virtual assistant, the electronic device 900 can minimize the first state window 60 to the navigation bar 61. The preset time length can be, for example, 10 seconds, and the embodiments of the present application do not limit the specific value of the preset time length.

[0631] If there is interaction content between the user and the virtual assistant before the virtual assistant exits, the first state window 60 can cache the interaction content to generate historical content. The first window 58 in other states can also cache the interaction content, and details are not described herein. The manner in which the virtual assistant exits based on the gesture operation has been described in the foregoing embodiments, and details are not described herein.

[0632] FIG. 51 is a second scenario in which the virtual assistant session ends according to an embodiment of the present application.

[0633] As shown in FIG. 51, the user can also exit the virtual assistant based on other operation forms. For example, the user can perform an upward swipe operation from the bottom of the display screen of the electronic device 900. In response to the upward swipe operation from the bottom of the display screen of the electronic device 900, the electronic device 900 returns to the home screen page 57 from the first application and ends the virtual assistant session.

[0634] If the user needs to use the virtual assistant again for an interaction process, the user can perform the wake-up operation again.

[0635] FIG. 52 is a scenario in which the virtual assistant is woken up again according to an embodiment of the present application.

[0636] As shown in A of FIG. 52, the user can wake up the virtual assistant again based on the navigation bar 61 in the first wake-up manner. If the user has interacted with the virtual assistant before and the virtual assistant has cached the interaction content, in response to the operation of the user waking up the virtual assistant again, the electronic device 900 displays the first sub-window 104 (or the second sub-window 105) corresponding to the virtual assistant and displays the historical content in the first sub-window 104.

[0637] As shown in B of FIG. 52, the user can wake up the virtual assistant again based on the navigation bar 61. Whether the user has interacted with the virtual assistant before or not, the electronic device 900 displays the first state window 60. That is, the electronic device 900 does not need to display the historical content. If the electronic device 900 is set in this form, the electronic device 900 does not need to cache the interaction content when the user ends the virtual assistant session in the foregoing embodiments.

[0638] The method shown in the embodiments of the present application supports the operation on the page in the first application again after the virtual assistant is woken up in the first application, that is, the first application and the virtual assistant can operate cooperatively, and the virtual assistant can provide a service recommendation matched with the updated page. The present application improves the user experience in the use of the virtual assistant.

[0639] The following exemplary describes a case in which a user inputs multi-modal content to a virtual assistant to make the virtual assistant update service recommendations.

[0640] The multi-modal content includes multiple different types of data modalities, including but not limited to visual modalities, auditory modalities, textual modalities, and sensor modalities, etc. The visual modalities are visual information obtained through image and video data, such as visual feature information extracted from images based on image recognition technology. The auditory modalities are auditory information obtained through audio and speech data, such as text information obtained by converting speech based on speech recognition technology. The textual modalities are information obtained through natural language text, such as text information obtained by classifying text using natural language processing technology. The sensor modalities are data information from various sensors built into electronic devices, such as temperature information, pressure information, and location information, etc.

[0641] In an implementation manner, the virtual assistant display method further includes steps S11-S13.

[0642] In step S11, the electronic device 900 displays the first page 87.

[0643] The following embodiments of the present application exemplarily illustrate a case in which the first application is a chat application.

[0644] FIG. 53 is a schematic diagram of a display manner of the first page according to an embodiment of the present application.

[0645] In the case where the first application is a chat application, the first page 87 has a display form as shown in FIG. 53.

[0646] The user can wake up the virtual assistant on the first page 87 to make the electronic device 900 provide services to the user based on the virtual assistant.

[0647] In step S12, in response to a long-press operation on the navigation button, or in response to a long-press operation on the power key, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first status window 60 in a region adjacent to the navigation button, and display the first control 59 in a region adjacent to the first status window 60.

[0648] At step S13, in response to sending the second target content on the first page 87 to the virtual assistant, the electronic device 900 changes from displaying the first status window 60 to displaying a second status window 62 on the first page 87, displays the second target content in the second status window 62, and displays at least one update control associated with the second target content in the second status window 62, wherein the update control is updated based on the first control 59, and each update control is configured to provide at least one service recommendation matching the second target content.

[0649] It should be noted that if the electronic device 900 has already displayed the second status window 62 on the first page 87, the electronic device 900 does not need to change from the first status window 60 to the second status window 62, and the electronic device 900 can directly display the target content in the second status window 62.

[0650] FIG. 54 is a schematic diagram of a first scenario in which a user sends a second target content to a virtual assistant according to an embodiment of the present application.

[0651] As shown in FIG. 54, the second target content 140 is exemplarily taken as a picture.

[0652] The user can perform an operation of long-pressing and dragging the second target content 140 on the first page 87 to the first status window 60. In response to the operation of long-pressing and dragging the second target content 140 to the first status window 60, the electronic device 900 sends the second target content 140 to the virtual assistant, changes from displaying the first status window 60 to displaying a second status window 62 on the first page 87, displays the second target content 140 in the second status window 62, and displays at least one first recommendation control 141 associated with the second target content 140 in the second status window 62.

[0653] When the electronic device 900 displays the second target content 140 in the second status window 62, the electronic device 900 synchronously uploads the second target content 140 to the first server, and does not display the uploading process of the second target content 140 in the second status window 62. In this way, the visual experience of the user can be improved.

[0654] In the embodiments of the present application, the first server includes a visual question answering (VQA) server. In practice, the first server can also include other types of servers, such as a natural language processing (NLP) server. The embodiments of the present application do not limit the specific type of the first server, and the embodiments of the present application only exemplarily illustrate the first server as a VQA server. VQA is an artificial intelligence technology that combines computer vision and natural language processing technology, and can understand image content and answer natural language questions related to images. The VQA server can implement image processing, question understanding, answer generation, model training and deployment, multi-modal data processing and other functions.

[0655] The specific way in which the update control 141 provides service recommendations is exemplarily illustrated below.

[0656] FIG. 55 is a first interaction diagram of a virtual assistant display method according to an embodiment of the present application.

[0657] As shown in FIG. 55, in one implementation, step S13 includes steps S131-S138.

[0658] Step S131, in response to sending the second target content 140 on the first page 87 to the virtual assistant, the virtual assistant acquires the second target content 140 based on a first program framework.

[0659] The first program framework can be part of the operating system of the electronic device 900, including multiple modules such as speech recognition, semantic understanding, and speech synthesis. The framework allows users to interact with the electronic device 900 through voice commands, keyboard instructions, and other means, thereby enabling the electronic device 900 to perform various services such as address lookup services and schedule addition services. In the present embodiment, the first program framework integrates service recommendation functions in a multi-modal scenario, i.e., after the user sends the second target content 140 on the first page 87 to the virtual assistant, the second target content 140 can be acquired and recognized without the user sending further instructions, and service recommendations can be provided based on the second target content 140.

[0660] Step S132, the first program framework calls a first content understanding service to perform content recognition processing on the second target content 140, and obtains a first content recognition result.

[0661] In one implementation, step S132 includes steps S1321-S1324.

[0662] Step S1321, the first program framework caches the acquired second target content 140 to a first program sub-framework therein.

[0663] The first program sub-framework can be a part of the first program framework and is a framework for service recommendation. It should be noted that the first program sub-framework can be independent of the first program framework, and the first program sub-framework can be based on the implementation function between the first program framework. The embodiments of the present application do not limit the specific setting mode of the first program sub-framework. The embodiments of the present application only exemplarily illustrate that the first program sub-framework is a part of the first program framework.

[0664] In step S1322, the first program sub-framework sends the second target content 140 to the first content understanding service.

[0665] In step S1323, the first content understanding service performs content recognition processing on the second target content 140 based on at least one deep learning model to obtain a first content recognition result.

[0666] The first content recognition result includes at least one of a scene type, a picture label, and an entity type corresponding to the second target content 140.

[0667] Specifically, the first content understanding service can be a multi-modal content understanding service deployed in a cloud server. The embodiments of the present application do not limit the specific service type of the first content understanding service.

[0668] The first content understanding service is exemplarily illustrated as a multi-modal content understanding service.

[0669] The multi-modal content understanding service is an artificial intelligence service that can process and analyze data of different modalities based on the deep learning model set therein and perform content recognition processing on the data. That is, the multi-modal content understanding service has multi-path recognition capability for the second target content 140.

[0670] In an implementation manner, the deep learning model can include at least one of a face detection model, an optical character recognition model, a target detection model, an image scene recognition model, and an address recognition model. The embodiments of the present application do not limit the number and type of deep learning models set in the first content understanding service.

[0671] Based on the deep learning model in the multi-modal content understanding service, the multi-modal content understanding service can perform content recognition processing on the second target content 140 to obtain the first content recognition result.

[0672] For example, the first content recognition result can be in a JavaScript Object Notation (JSON) data format. The first content recognition result can also be in other forms of data format, and the embodiments of the present application do not limit the data format of the first content recognition result.

[0673] In step S1324, the first content understanding service sends the first content recognition result to the first program sub-framework.

[0674] It should be noted that after the first program sub-framework receives the first content recognition result, the second target content 140 can be post-processed.

[0675] The post-processing refers to a process of further processing the second target content 140 by a script or a program after the second target content 140 is loaded and parsed, so that specific functions can be implemented or user experience can be optimized.

[0676] In the embodiments of the present application, the first program sub-framework can detect the playing state of the video when the second target content 140 includes the video, so as to ensure that the user intent obtained by subsequent intent prediction is more accurate.

[0677] The first program sub-framework can also detect the fill-in information in the picture when the second target content 140 includes the picture, so as to ensure that the user intent obtained by subsequent intent prediction is more accurate.

[0678] The embodiments of the present application can provide different post-processing modes for different second target content 140, and the embodiments of the present application do not limit the specific mode of post-processing.

[0679] In step S133, the first program framework sends the first content recognition result to a first service plug-in in the first perception service.

[0680] The first perception service can be a service of obtaining information by using sensors and perception technology. The first perception service can exist in the form of an Android application package (APK), and can also exist in the form of an operating system service, a system-level software, a micro-service or other forms, and the embodiments of the present application do not limit the specific form of the first perception service.

[0681] In an implementation manner, the first service plug-in can be an intent prediction plug-in, which is used for user intent prediction for the first content recognition result. The embodiments of the present application do not limit the plug-in type of the first service plug-in.

[0682] The intent prediction plug-in is a software component for identifying and predicting user intent, which can implement natural language understanding, intent recognition, entity extraction, context management, and the like.

[0683] In step S134, the first service plug-in matches the first content recognition result with the first preset configuration to obtain at least one first target user intent corresponding to the first content recognition result, wherein the first preset configuration includes at least one user intent.

[0684] In an example, the first service plug-in can obtain the first preset configuration containing multiple user intents from the cloud server during an idle period of the electronic device 900. For example, the first service plug-in can obtain the first preset configuration when the electronic device 900 is charging and the screen is off. The embodiments of the present application do not limit the specific manner in which the first service plug-in obtains the first preset configuration.

[0685] In an example, the first target user intent can be in JSON data format. The first target user intent can also be in other forms of data format, and the embodiments of the present application do not limit the data format of the first target user intent.

[0686] In step S135, the first service plug-in sends the first target user intent to the first program framework.

[0687] In step S136, the first program framework sends the first target user intent to the second service plug-in in the first perception service.

[0688] In step S137, the second service plug-in matches the first target user intent with the second preset configuration to obtain a first service recommendation corresponding to the first target user intent, wherein the second preset configuration includes at least one service recommendation, and the first service recommendation is a service recommendation matched with the second target content 140.

[0689] In an example, the second service plug-in can be a service recommendation plug-in for service recommendation for the first target user intent. The embodiments of the present application do not limit the plug-in type of the second service plug-in.

[0690] The service recommendation plug-in is a software component for service recommendation for user intent, which can implement functions such as decision support and optimization of resource allocation.

[0691] In an example, the second service plug-in can obtain the second preset configuration containing multiple service recommendations from the cloud server during an idle period of the electronic device 900. For example, the first service plug-in can obtain the first preset configuration when the electronic device 900 is charging and the screen is off. The embodiments of the present application do not limit the specific manner in which the second service plug-in obtains the second preset configuration.

[0692] It should be noted that the first preset configuration and the second preset configuration can be deployed in different services of the same cloud server, or can be deployed in different cloud servers respectively. For example, the first preset configuration can be deployed in the decision cloud platform, and the second preset configuration can be deployed in the cloud server corresponding to the first program framework. The specific deployment mode of the first preset configuration and the second preset configuration is not limited in the embodiments of the present application.

[0693] In an implementation manner, the step S137 includes steps S1371-S1372.

[0694] FIG. 56 is a schematic diagram of a scenario in which the second service plug-in performs recall processing and sorting processing according to an embodiment of the present application.

[0695] As shown in FIGS. 55 and 56, in step S1371, the second service plug-in performs recall processing on the first target user intent to obtain a target service recommendation corresponding to the first target user intent, wherein the recall processing is used to filter the service recommendations in the second preset configuration.

[0696] In an implementation manner, the specific mode of the recall processing in the embodiment is multi-path recall processing, which is used to quickly filter out the preferred content of the user from a large amount of candidate content. Specifically, the multi-path recall processing can involve multiple different recall strategies. Each strategy can recall content from different perspectives or using different features. After obtaining multiple recall contents, the contents can be merged to form a candidate set for subsequent sorting processing.

[0697] In the embodiment of the present application, the cloud server can be deployed with a recall service, which can include a personal memory recall module, a service management framework module, and an operation data recall module. Each module can correspond to a different recall strategy.

[0698] The personal memory recall module can be used to recall content according to the historical behavior and preferences of the user. This module relies on user portraits and historical data to recall the user's historical interest points and then recommend similar or related new content.

[0699] The service management framework module is deployed with multiple service recommendations corresponding to user intents, and is the main module in the recall service, which is used to implement related intent recall.

[0700] The operation data recall module can be used to recall content according to the data collected during the operation. These data include user feedback, behavior statistics, market trends, popular searches, etc. The operation data recall module can discover the current popular content or service, and the trend that meets the current market and user demand, so as to provide service recommendations that are more in line with the current interests of the user.

[0701] It should be noted that the recall service can also include modules with other recall strategies, and the embodiments of the present application do not limit this.

[0702] In this way, the second service plug-in can recall the first target user intention based on the interaction with the recall service to obtain a target service recommendation corresponding to the first target user intention.

[0703] In step S1372, the second service plug-in sorts the target service recommendation to obtain a first service recommendation.

[0704] Since the first target user intention usually corresponds to multiple target service recommendations, the second service plug-in needs to sort these target service recommendations.

[0705] In the embodiments of the present application, the cloud server can be deployed with a sorting service. The sorting service can score and sort the target service recommendations obtained based on the recall service based on a certain algorithm to determine the order of display to the user. In this way, the sorting service can optimize the user experience and improve the relevance and accuracy of the target service recommendation and the second target content 140.

[0706] The specific way in which the sorting service sorts the target service recommendation is described below.

[0707] The service recommendation includes a text recognition service recommendation, an object recognition service recommendation, and a screen content service recommendation.

[0708] The text recognition service recommendation is a service recommendation provided for text or textual information in the picture.

[0709] Specifically, the text recognition service recommendation includes a whole picture-based service recommendation and a text content-based service recommendation.

[0710] The whole picture-based service recommendation includes a limited service recommendation and a general service recommendation. The limited service recommendation includes a "document scanning", a "card scanning", a "test paper restoration", a "privacy coding" and the like. The general service recommendation includes a "translation" service recommendation.

[0711] The text content-based service recommendation includes an entity service recommendation, a limited service recommendation and a general service recommendation. The entity service recommendation includes a "view address", a "call", a "send email", an "add to address book" and the like. The entity service recommendation can be a service recommendation for jumping to a third-party application. The limited service recommendation includes a "create schedule", a "form extraction" and the like. The general service recommendation includes a "text extraction" service recommendation.

[0712] The object recognition service recommendation is a service recommendation provided for objects in the picture.

[0713] Specifically, the object-based service recommendation includes a whole-picture-based service recommendation and a subject-based service recommendation.

[0714] The whole-picture-based service recommendation includes a limited-service recommendation and a general-service recommendation. The limited-service recommendation includes a "card certificate extraction related" service recommendation, a "perfect portrait" service recommendation, a "remove passerby" service recommendation, a "generate ID photo" service recommendation, and a "one-key beautification" service recommendation. The general-service recommendation includes an "AI stylization" service recommendation and an "add time and place stickers" service recommendation.

[0715] The subject-based service recommendation includes a VQA service recommendation and a general-service recommendation. The VQA service recommendation includes a "who is this" service recommendation, a "what is this" service recommendation, and a "where is this" service recommendation. The VQA service recommendation is a service recommendation that can be used to interact with a VQA server to provide VQA information. The general-service recommendation includes an "intelligent cutout" service recommendation and a "text matching" service recommendation.

[0716] The screen content-based service recommendation includes a content source service recommendation, a scene-based service recommendation, and a screen content general service recommendation.

[0717] The content source service recommendation is a service recommendation provided by the electronic device 900 after determining that the current screen content is a content source such as a document, an information stream, or a large picture. The content source service recommendation includes a "document summary" service recommendation, a "generate abstract" service recommendation, and a "remove passerby" service recommendation.

[0718] The scene-based service recommendation includes a "magic cube customization" service recommendation, an "AI face swap detection" service recommendation, an "AI caption" service recommendation, an "intelligent form filling" service recommendation, and a "privacy coding" service recommendation.

[0719] The screen content general service recommendation includes a "full-screen translation" service recommendation, a "text extraction" service recommendation, and an "intelligent cutout" service recommendation.

[0720] Based on the above-mentioned various service recommendations, the ranking service is provided with corresponding ranking rules.

[0721] In an implementation manner, the ranking service provides a general ranking rule.

[0722] Specifically, when the second target content 140 is a picture of pure text or pure object, the ranking service can be ranked according to a preset priority, where the priority setting manner can be that the entity service recommendation is prior to the limited-service recommendation, the limited-service recommendation is prior to the VQA service recommendation, and the VQA service recommendation is prior to the general-service recommendation.

[0723] In the priority sorting process, since the limited-class service recommendation corresponds to the service recommendation based on the whole picture and the service recommendation based on the text content, in the limited-class service recommendation, the service recommendation based on the whole picture is prior to the service recommendation based on the text content. In the entity-class service recommendation, the entity service recommendations of different categories can be sorted according to the display order in the display screen. In the limited-class service recommendation, the sorting can be performed according to the condition requirements from high to low. The condition requirements can be preset in the electronic device 900. In the general-class service recommendation, the sorting can be performed according to the condition requirements from high to low.

[0724] If the language in the second target content 140 is not equal to the system language based on the Optical Character Recognition (OCR) model, the priority of the "translation" service recommendation in the general-class service recommendation is set to be higher than the priority of the entity-class service recommendation. If the language in the second target content 140 is equal to the system language based on the OCR model, the "translation" service recommendation is placed last.

[0725] When the second target content 140 is a picture including both text and objects, the sorting service can determine the priority sorting according to the proportion of the text and the objects.

[0726] If the proportion of the text is greater than the proportion of the objects, the non-general-class service recommendation corresponding to the text is displayed first, the non-general-class service recommendation corresponding to the picture is displayed second, the general-class service recommendation corresponding to the text is displayed third, and the general-class service recommendation corresponding to the picture is displayed last.

[0727] The non-general-class service recommendation includes the entity-class service recommendation, the limited-class service recommendation, and the VQA-class service recommendation.

[0728] If the proportion of the objects is greater than the proportion of the text, the non-general-class service recommendation corresponding to the objects is displayed first, the non-general-class service recommendation corresponding to the text is displayed second, the general-class service recommendation corresponding to the objects is displayed third, and the general-class service recommendation corresponding to the text is displayed last.

[0729] In this embodiment, the general sorting rule can be used for the second target content 140 dragged by the user into the first state window 60, to obtain the sorted target service recommendation, that is, the first service recommendation.

[0730] It should be noted that other sorting rules are also set in the sorting service, for example, the sorting rule based on the scene and the content can be used for the first control 59 in the foregoing embodiment.

[0731] Since the first control 59 is generated by the electronic device 900 in page recognition of the first page 87 without user input, the ranking service can perform the scene and content-based ranking rule based on the current scene and page content of the first page 87.

[0732] The scene and content-based ranking rule can limit the displayed service recommendations.

[0733] For example, if the electronic device 900 identifies that the first page 87 is in a large image page, or has a scene of text that cannot be copied, the ranking service can provide a "text extraction" service recommendation, and if it is not in the above-mentioned scene, the ranking service cannot display the "text extraction" service recommendation.

[0734] The scene and content-based ranking rule can prioritize the content source-based service recommendation in the same type of service.

[0735] For example, if the electronic device 900 identifies that the first page 87 includes an image, the ranking service prioritizes the "image translation" service recommendation over the "full-screen translation" service recommendation.

[0736] In addition, the ranking service can also prioritize service recommendations that do not involve storage and can be covered by general service recommendations, such as the "text extraction" service recommendation.

[0737] The scene and content-based ranking rule can prioritize scene-based service recommendations in non-content source pages, followed by text recognition service recommendations and object recognition service recommendations, and not recommend "test paper restoration", "document correction" and other services based on the entire image.

[0738] Step S138, the second service plug-in sends the first service recommendation to the first program framework, so that the first program framework displays the first service recommendation to the update control 141.

[0739] It should be noted that since the push of the update control 141 requires a certain time length, the electronic device 900 displays the update control 141 after displaying the second target content 140 in the second state window 62 for a first time length. During the first time length, the user can send instructions to the virtual assistant based on voice input or keyboard input.

[0740] It should also be noted that the electronic device 900 does not obtain the first service recommendation at the same time, because the electronic device 900 has different recognition capabilities for different contents in the second target content 140, and the provided services are different, so the time length of obtaining the first service recommendation is different. For example, the electronic device 900 can first obtain the first service recommendation provided for text, and then obtain the first service recommendation provided for pictures. In this way, the electronic device 900 displays the first service recommendation obtained first in priority. If the priority of the first service recommendation obtained subsequently is higher than the priority of the first service recommendation obtained first, the electronic device 900 can adjust the arrangement order of the first service recommendation based on the dynamic effect. For example, based on the squeezing dynamic effect, the electronic device 900 squeezes the first service recommendation obtained subsequently to the first service recommendation obtained first, so as to display the first service recommendation obtained subsequently in priority.

[0741] In this way, the user sends the second target content 140 on the first page 87 to the virtual assistant, and the virtual assistant directly displays the second target content 140 in the second state window 62 without displaying the uploading process of the second target content 140, thereby improving the visual experience of the user.

[0742] FIG. 57 is a first scenario diagram of interaction between a user and a virtual assistant based on an update control according to an embodiment of the present application.

[0743] As shown in FIG. 57, in an implementation, when the second target content 140 is a picture, steps S13 further include steps S14-S16.

[0744] Step S14, in response to the clicking operation on the first target control in the update control 141, the electronic device 900 displays the first target service recommendation 142 corresponding to the first target control in the second state window 62, and uploads the first target service recommendation 142 to the VQA server.

[0745] Step S15, during the uploading of the first target service recommendation 142 to the VQA server, the electronic device 900 displays the first response dialog box 143 corresponding to the first target service recommendation 142 in the second state window 62, and displays the first waiting information 144 in the first response dialog box 143. The first waiting information 144 is used to prompt the user that the electronic device is in the process of obtaining the first VQA result 145 corresponding to the first target service recommendation.

[0746] In the embodiment of the present application, the first waiting information 144 can be displayed in the form of a dynamic effect as shown in FIG. 57, or in the form of text, for example, displayed as “Obtaining…”. The specific display form of the first waiting information 144 is not limited in the embodiment of the present application.

[0747] The first target service recommendation 142 is actually a VQA instruction. In the process of sending the VQA instruction, if the second target content 140 is not uploaded completely, the uploading progress of the second target content 140 can be integrated into the first waiting information 144, so that the visual experience of the user is improved.

[0748] In step S16, after the electronic device 900 obtains the first VQA result 145 sent by the VQA server for the first target service recommendation 142, the electronic device 900 displays the first VQA result 145 in the first response dialogue box 143.

[0749] It should be noted that the first VQA result 145 includes a generating state and a completely generated state as shown in FIG. 57.

[0750] In this way, the user quickly interacts with the virtual assistant based on the update control 141 to use the service provided by the service recommendation in the update control 141. The method shown in the embodiments of the present application improves the user experience.

[0751] The above embodiments show a scenario in which the user successfully quickly interacts with the virtual assistant based on the update control 141. Actually, in some cases, there is an abnormal scenario in which the interaction between the user and the virtual assistant fails.

[0752] FIG. 58 is a schematic diagram of a first abnormal scenario of interaction between the user and the virtual assistant according to an embodiment of the present application.

[0753] As shown in FIG. 57 and FIG. 58, in an implementation, after step S15, steps S17-S19 are further included in the case of VQA server exception or network exception.

[0754] In step S17, when the first VQA result 145 sent by the VQA server for the first target service recommendation 142 is not obtained, the electronic device 900 displays first error information 146 in the first response dialogue box 143, and displays a first error button 147 in a region adjacent to the first target service recommendation 142. The first error information 146 is used to prompt the user that the VQA server is abnormal.

[0755] In the embodiments of the present application, the first error information 146 can be displayed in the form of text, for example, displayed as “server exception, please try again later”. The first error information 146 can also be displayed in the form of dynamic effect, and the specific display form of the first error information 146 is not limited in the embodiments of the present application.

[0756] In step S18, in response to the click operation on the first error button 147, the electronic device 900 stops displaying the first error button 147, and displays the first waiting information 144 in the first response dialogue box 143.

[0757] In this way, the first waiting information 144 can cover the first error reporting information 146, and the display manner improves the visual experience of the user.

[0758] At step S19, after obtaining the first VQA result 145 recommended by the VQA server for the first target service recommendation 142, the electronic device 900 displays the first VQA result 145 in the first response dialogue box 143.

[0759] That is, in the embodiment of the application, the electronic device 900 supports the text instruction resending mechanism, so that the user can quickly resend the VQA instruction based on the first error reporting button 147, thereby improving the user experience.

[0760] FIG. 59 is a schematic diagram of a second abnormal scenario of user interaction with a virtual assistant according to an embodiment of the application.

[0761] As shown in FIGS. 57 and 59, in an implementation manner, after step S15, steps S20-S22 are further included.

[0762] At step S20, when the first VQA result 145 recommended by the VQA server for the first target service recommendation 142 is not obtained, the electronic device 900 displays the first error reporting information 146 in the first response dialogue box 143, and displays the first error reporting button 147 in a region adjacent to the first target service recommendation 142.

[0763] At step S21, in response to inputting the first instruction 148 into the second state window 62, the electronic device 900 stops displaying the first error reporting button 147, displays a second response dialogue box 149 corresponding to the first instruction 148 in the second state window 62, and displays second waiting information 150 in the second response dialogue box 149, where the second waiting information 150 is used to prompt the user that the electronic device 900 is in a process of obtaining a first response result 151 corresponding to the first instruction 148.

[0764] The user can input a voice instruction or a text instruction into the second state window 62 to make the electronic device 900 generate the first instruction 148. The specific operation form of the user inputting the first instruction 148 is not limited in the embodiment of the application.

[0765] At step S22, after obtaining the first response result 151, the electronic device 900 displays the first response result 151 in the second response dialogue box 149.

[0766] In the embodiment of the application, after receiving a new instruction (for example, the first instruction 148), the electronic device 900 no longer supports the first target service recommendation 142 of resending the error reporting.

[0767] In another implementation, the electronic device 900 can also support resending the error reporting first target service recommendation 142 after receiving a new instruction (e.g., the first instruction 148). For example, during the process of steps S20-S21, the electronic device 900 continuously displays the first error reporting button 147, and the user can perform a click operation on the first error reporting button 147. In response to the click operation on the first error reporting button 147, the electronic device 900 stops displaying the first error reporting button 147 and adds a response dialog box.

[0768] FIG. 60 is a third abnormal scenario diagram of user interaction with the virtual assistant according to an embodiment of the present application.

[0769] As shown in FIG. 60, in an implementation, when the second target content 140 is a risky picture, step S12 is followed by step S23.

[0770] Step S23, in response to sending the second target content 140 on the first page 87 to the virtual assistant, the electronic device 900 changes the display of the first state window 60 on the first page 87 to the display of the second state window 62, displays the first shielding picture 152 in the second state window 62, and displays the shielding error information 153 in the second state window 62.

[0771] For example, the text content of the shielding error information 153 can be “This picture is shielded due to risk”. The specific display content of the shielding error information 153 is not limited in the present application.

[0772] The electronic device 900 does not support resending the second target content 140 when it is determined that the second target content 140 is a risky picture.

[0773] FIG. 61 is a second scenario diagram of user sending target content to the virtual assistant according to an embodiment of the present application.

[0774] As shown in FIG. 61, in an implementation, the method includes steps S31-S34.

[0775] Step S31, the electronic device 900 displays the first page 87, and the first page 87 is a home screen page.

[0776] The user can wake up the virtual assistant on the first page 87 to make the electronic device 900 provide services to the user based on the virtual assistant.

[0777] In step S32, in response to the long-press operation on the navigation button, or in response to the long-press operation on the power key, the electronic device 900 wakes up the virtual assistant on the first page 87 to display the first status window 60 in a region adjacent to the navigation button, and display the first accessory region 154 in a region adjacent to the first status window 60, where the first accessory region 154 includes at least one accessory entry 155 for providing the second target content 140.

[0778] The embodiment takes the navigation button as the navigation bar 61 for example.

[0779] In the embodiment, the accessory entry 155 includes a photo accessory entry, a photo shooting accessory entry, and a document accessory entry, where the photo accessory entry and the photo shooting accessory entry can be used to provide picture target content, and the document accessory entry can be used to provide document second target content 140. The number and type of the accessory entry 155 are not limited in the embodiment.

[0780] In step S33, in response to the click operation on a target accessory entry in the accessory entry 155, the electronic device 900 displays an accessory page 156 corresponding to the target accessory entry, and displays the second target content 140 in the accessory page 156.

[0781] The embodiment takes the target accessory entry as the document accessory entry for example. In the case where the target accessory entry is the document accessory entry, the second target content 140 is a document.

[0782] In step S34, in response to the selection operation on the second target content 140, the electronic device 900 sends the second target content 140 to the virtual assistant to display the second status window 62 on the first page 87, and display the second target content 140 in the second status window 62, and display at least one update control 141 associated with the second target content 140 in the second status window 62.

[0783] When the electronic device 900 displays the second target content 140 in the second status window 62, the electronic device 900 synchronously uploads the second target content 140 to the second server, and does not display the uploading process of the second target content 140 in the second status window 62. In this way, the visual experience of the user can be improved.

[0784] In the embodiments of the present application, the second server includes a document question answering (DQA) server. In practice, the second server can also include other types of servers, such as a natural language processing (NLP) server. The embodiments of the present application do not limit the specific type of the second server, and the embodiments of the present application are only exemplarily described by taking the second server as a DQA server.

[0785] DQA is an information retrieval technology that focuses on extracting specific information from unstructured documents to answer user questions. DQA is used to process textual information in documents. The DQA server can implement functions such as information retrieval, optical character recognition, natural language processing, and machine reading comprehension.

[0786] The manner in which the update control 141 provides service recommendations can refer to the foregoing embodiments, and the present application does not repeat the same.

[0787] FIG. 62 is a schematic diagram of a second scenario in which a user interacts with a virtual assistant based on an update control according to an embodiment of the present application.

[0788] As shown in FIG. 62, in the case where the second target content 140 is a document, steps S35-S37 are further included after step S34.

[0789] In step S35, in response to a click operation on the second target control in the update control 141, the electronic device 900 displays a second target service recommendation 157 corresponding to the second target control in the second state window 62, and uploads the second target service recommendation 157 to a DQA server.

[0790] In step S36, during the uploading of the second target service recommendation 157 to the DQA server, the electronic device 900 displays a third response dialog box 158 corresponding to the second target service recommendation 157 in the second state window 62, and displays third waiting information 159 in the third response dialog box 158. The third waiting information 159 is used to prompt the user that the electronic device is in the process of obtaining a first DQA result 160 corresponding to the second target service recommendation 157.

[0791] In the embodiment of the present application, the third waiting information 159 can have various display forms. For example, when the waiting time is less than or equal to 5 seconds, the third waiting information 159 can be displayed as "parsing the document", when the waiting time is greater than 5 seconds and less than or equal to 15 seconds, the third waiting information 159 can be displayed as "reading the content", and when the waiting time is greater than 15 seconds, the third waiting information 159 can be displayed as "analyzing the content". In this way, the various display forms of the third waiting information 159 can alleviate user anxiety and improve user experience. The third waiting information 159 can also be displayed in the form of a dynamic effect. The specific display form of the third waiting information 159 is not limited in the embodiment of the present application. The attached drawings of the present embodiment only exemplarily illustrate one of the display forms of the third waiting information 159.

[0792] In the embodiment of the present application, the third waiting information 159 can have various display forms. For example, when the waiting time is less than or equal to 5 seconds, the third waiting information 159 can be displayed as "parsing the document", when the waiting time is greater than 5 seconds and less than or equal to 15 seconds, the third waiting information 159 can be displayed as "reading the content", and when the waiting time is greater than 15 seconds, the third waiting information 159 can be displayed as "analyzing the content". In this way, the various display forms of the third waiting information 159 can alleviate user anxiety and improve user experience. The third waiting information 159 can also be displayed in the form of a dynamic effect. The specific display form of the third waiting information 159 is not limited in the embodiment of the present application. The attached drawings of the present embodiment only exemplarily illustrate one of the display forms of the third waiting information 159.

[0793] In the embodiment of the present application, the third waiting information 159 can have various display forms. For example, when the waiting time is less than or equal to 5 seconds, the third waiting information 159 can be displayed as "parsing the document", when the waiting time is greater than 5 seconds and less than or equal to 15 seconds, the third waiting information 159 can be displayed as "reading the content", and when the waiting time is greater than 15 seconds, the third waiting information 159 can be displayed as "analyzing the content". In this way, the various display forms of the third waiting information 159 can alleviate user anxiety and improve user experience. The third waiting information 159 can also be displayed in the form of a dynamic effect. The specific display form of the third waiting information 159 is not limited in the embodiment of the present application. The attached drawings of the present embodiment only exemplarily illustrate one of the display forms of the third waiting information 159.

[0794] It should be noted that the first DQA result 145 includes a generating state and a complete generating state as shown in FIG. 19.

[0795] In this way, the user quickly interacts with the virtual assistant based on the update control 141 to use the services provided by the service recommendation in the update control 141. The method shown in the embodiment of the present application improves the user experience.

[0796] It should be noted that in the foregoing embodiment, the user can also send the document type second target content 140 to the virtual assistant in a drag and drop manner after waking up the virtual assistant. The subsequent implementation manner can refer to the present embodiment, and the embodiments of the present application can be mutually referred to, and the present application will not be described here.

[0797] The foregoing embodiment shows a successful scenario of the user quickly interacting with the virtual assistant based on the update control 141. Actually, in some cases, there is an abnormal scenario of interaction failure between the user and the virtual assistant.

[0798] FIG. 63 is a schematic diagram of a fourth abnormal scenario of user interaction with a virtual assistant according to an embodiment of the present application.

[0799] As shown in FIGS. 62 and 63, in an implementation, after step S36, steps S38-S40 are further included in the case of DQA server exception or network exception.

[0800] Step S38, when the first DQA result 160 issued by the DQA server for the second target service recommendation 157 is not acquired, the electronic device 900 displays second error information 161 in the third response dialog box 158, and displays a second error button 162 adjacent to the second target service recommendation 157, the second error information 161 is used to prompt the user that the DQA server is abnormal.

[0801] In the embodiment of the application, the second error information 161 can be displayed in the form of text, for example, displayed as “server exception, please try later”. The second error information 161 can also be displayed in the form of dynamic effect, and the specific display form of the second error information 161 is not limited in the embodiment of the application.

[0802] Step S39, in response to the click operation on the second error button 162, the electronic device 900 stops displaying the second error button 162, and displays the third waiting information 159 in the third response dialog box 158.

[0803] In this way, the third waiting information 159 can cover the second error information 161, and this display mode improves the visual experience of the user.

[0804] Step S40, after the first DQA result 160 issued by the DQA server for the second target service recommendation 157 is acquired, the electronic device 900 displays the first DQA result 160 in the third response dialog box 158.

[0805] That is, in the embodiment of the application, the electronic device 900 supports the text instruction retransmission mechanism, so that the user can quickly resend the DQA instruction based on the second error button 162, and the use experience of the user is improved.

[0806] FIG. 64 is a schematic diagram of a fifth exception scenario of user interaction with a virtual assistant provided by an embodiment of the application.

[0807] As shown in FIG. 64, in an implementation, after step S36, steps S41-S43 are further included.

[0808] Step S41, when the first DQA result 160 issued by the DQA server for the second target service recommendation 157 is not acquired, the electronic device 900 displays second error information 161 in the third response dialog box 158, and displays a second error button 162 adjacent to the second target service recommendation 157.

[0809] Step S42, in response to the operation of inputting the second instruction 163 into the second state window 62, the electronic device 900 stops displaying the second error reporting button 162, displays a fourth response dialog box 164 corresponding to the second instruction 163 in the second state window 62, and displays fourth waiting information 165 in the fourth response dialog box 164, where the fourth waiting information 165 is used to prompt the user that the electronic device 900 is in the process of obtaining a second response result 166 corresponding to the second instruction 163.

[0810] In this embodiment of the present application, the user can input a voice instruction or a text instruction into the second state window 62 to make the electronic device 900 generate the second instruction 163. The specific operation form of the user inputting the second instruction 163 is not limited in this embodiment of the present application.

[0811] Step S43, after obtaining the second response result 166, the electronic device 900 displays the second response result 166 in the fourth response dialog box 164.

[0812] That is to say, in this embodiment of the present application, the electronic device 900 no longer supports the second target service recommendation 157 of re-sending error reporting after receiving a new instruction (for example, the second instruction 163).

[0813] FIG. 65 is a schematic diagram of a sixth abnormal scenario of the user interacting with the virtual assistant according to an embodiment of the present application.

[0814] As shown in FIG. 65, in one implementation, in the case that the second target content 140 is a document type that is not supported, step S34 further includes step S44.

[0815] Step S44, in response to the clicking operation on the second target control in the update control 141, the electronic device 900 displays the second target service recommendation 157 corresponding to the second target control in the second state window 62, and displays the parsing error information 167.

[0816] For example, the text content of the parsing error information 167 can be "temporary support for only image content documents". The specific display content of the parsing error information 167 is not limited in this embodiment of the present application.

[0817] In the case that the electronic device 900 determines that the second target content 140 is a document type that is not supported, the electronic device 900 does not support re-sending the second target content 140.

[0818] In another embodiment, in step S33, the target attachment entrance can be a picture attachment entrance, and in the case that the target attachment entrance is a picture attachment entrance, the second target content 140 can be a picture.

[0819] If the user performs a selection operation on multiple pictures, and the pictures include a picture that can be recognized by the electronic device 900, the electronic device 900 can display only the first picture in the second status window 62, display an abnormality prompt information in the second status window 62, and display an update control 141 corresponding to the first picture in the second status window 62. The text content of the abnormality prompt information can be, for example, "Multiple files are not supported at present, and the following services can be provided for the first picture".

[0820] If the user performs a selection operation on multiple pictures, and the pictures do not include a picture that can be recognized by the electronic device 900, the electronic device 900 can pop up a prompt control and display abnormality prompt information in the prompt control. The text content of the abnormality prompt information can be, for example, "Unsupported file type".

[0821] If the picture selected by the user is in a format that is not supported by the electronic device 900, the electronic device 900 pops up a prompt control and displays abnormality prompt information in the prompt control. The text content of the abnormality prompt information can be, for example, "Unsupported picture format".

[0822] If the picture selected by the user is larger than the upper limit supported by the electronic device 900, the electronic device 900 pops up a prompt control and displays abnormality prompt information in the prompt control. The text content of the abnormality prompt information can be, for example, "Pictures larger than 30 MB are not supported at present".

[0823] If the resolution of the picture selected by the user is larger than the upper limit supported by the electronic device 900, the electronic device 900 pops up a prompt control and displays abnormality prompt information in the prompt control. The text content of the abnormality prompt information can be, for example, "Picture resolution exceeds the limit".

[0824] In another embodiment, if the user performs a selection operation on multiple documents, and the documents include a document that can be parsed by the electronic device 900, the electronic device 900 can display only the first document in the second status window 62, display abnormality prompt information in the second status window 62, and display an update control 141 corresponding to the first document in the second status window 62. The text content of the abnormality prompt information can be, for example, "Multiple files are not supported at present, and the following services can be provided for the first document".

[0825] If the user performs a selection operation on multiple documents, and the documents do not include a document that can be parsed by the electronic device 900, the electronic device 900 can pop up a prompt control and display abnormality prompt information in the prompt control. The text content of the abnormality prompt information can be, for example, "Unsupported file type".

[0826] If the document selected by the user is in a format that is not supported by the electronic device 900, the electronic device pops up a prompt control and displays abnormal prompt information in the prompt control. The text content corresponding to the abnormal prompt information may be, for example, “word, pdf, and txt formats are not supported at present”.

[0827] If the document selected by the user is larger than the upper limit supported by the electronic device 900, the electronic device pops up a prompt control and displays abnormal prompt information in the prompt control. The text content corresponding to the abnormal prompt information may be, for example, “documents larger than 10 MB are not supported at present”.

[0828] If the number of documents uploaded by the user exceeds the upper limit supported by the electronic device 900, the electronic device pops up a prompt control and displays abnormal prompt information in the prompt control. The text content corresponding to the abnormal prompt information may be, for example, “the upper limit of the number of documents uploaded per day is 100”.

[0829] FIG. 66 is a schematic diagram of a third scenario in which a user sends second target content to a virtual assistant according to an embodiment of the present application.

[0830] As shown in FIG. 66, in one implementation, the method includes the following steps S51-S53.

[0831] In step S51, the electronic device 900 displays a first page 87.

[0832] The first page 87 is taken as an application page in a chat application program, and the second target content 140 on the first page 87 is taken as a picture of a person for exemplary description. In the case where the second target content 140 is a picture, the user can perform a long press operation on the second target content 140 so that the second target content 140 can be further subjected to a drag operation.

[0833] In step S52, in the case where the second target content 140 on the first page 87 can be subjected to a drag operation, in response to a long press operation on the second target content 140, the electronic device 900 displays a first screen area 168 and displays a first icon 169 corresponding to a virtual assistant in the first screen area 168.

[0834] In the present embodiment, the first screen area 168 is actually a kind of screen hot area, which can be set at the two side edges of the display screen. The user can long press a text, a picture, a video, or a file and drag the content to any one of the side edges of the display screen to trigger the first screen area 168. The user can also trigger the first screen area 168 by a screen capture or screen recognition operation. The specific manner of triggering the first screen area 168 is not limited in the present embodiment.

[0835] Step S53, in response to the operation of dragging the second target content 140 to the first icon 169, the electronic device 900 sends the second target content 140 to the virtual assistant to display the second status window 62 on the first page 87, and display the second target content 140 in the second status window 62, and display at least one update control 141 associated with the second target content 140 in the second status window 62.

[0836] FIG. 67 is a schematic diagram of a first scenario of interaction between a user and a virtual assistant based on second target content according to an embodiment of the present application.

[0837] As shown in FIG. 67, in an implementation, in the case that the second target content 140 is a picture, after step S53, steps S54-S55 are further included.

[0838] Step S54, in response to the click operation on the second target content 140, the electronic device 900 displays the first zoom-in page 170, and displays the second target content 140 in the zoom-in state on the first zoom-in page 170, and displays the first edit control 171, the first share control 172 and the first save control 173 on the first zoom-in page 170.

[0839] The first edit control 171 can be used to edit the second target content 140 in the zoom-in state, the first share control 172 can be used to share the second target content 140, and the first save control 173 can be used to save the second target content 140.

[0840] In an implementation, after the electronic device 900 displays the first zoom-in page 170 for a certain time length, the electronic device 900 can display the first zoom-in page 170 in an immersive state, i.e., hide the menu bar on the page for displaying the first edit control 171, the first share control 172 and the first save control 173.

[0841] The function of the electronic device 900 displaying the first zoom-in page 170 in the immersive state can be adapted to the function provided by the gallery application of the electronic device 900.

[0842] In an implementation, in response to the double-click operation on the second target content 140 in the zoom-in state, or in response to the double-finger pinch operation on the second target content 140 in the zoom-in state, the electronic device 900 performs zooming processing on the second target content 140 in the zoom-in state.

[0843] It should be noted that the function of the electronic device 900 performing zooming processing on the second target content 140 in the zoom-in state based on the gesture operation is also adapted to the function provided by the gallery application of the electronic device 900.

[0844] In an implementation manner, in response to the long-press operation on the second target content 140 in the zoom-in state, the electronic device 900 displays the first screen area 168.

[0845] In an implementation manner, in the immersive state, in response to the click operation on the second target content 140 in the zoom-in state, the electronic device 900 switches to the non-immersive state. And, in response to the click operation on the second target content 140 in the zoom-in state again, the electronic device 900 switches from the non-immersive state to the immersive state.

[0846] In an implementation manner, in response to the double-finger drag operation on the second target content 140 in the zoom-in state, the electronic device 900 can move the second target content 140.

[0847] In an implementation manner, in the case that the second target content 140 in the zoom-in state has text, in response to the long-press operation on the second target content 140, the electronic device 900 can preferentially select the text in the second target content 140.

[0848] In an implementation manner, in the case that the second target content 140 in the zoom-in state does not have text, in response to the long-press operation on the second target content 140, the electronic device 900 can display the second target content 140 as a draggable state.

[0849] In an implementation manner, in response to the downward sliding operation in the first zoom-in page 170, the electronic device 900 exits the first zoom-in page 170.

[0850] In an implementation manner, in response to the upward sliding operation in the first zoom-in page 170, or, in response to the leftward sliding operation in the first zoom-in page 170, or, in response to the downward sliding operation in the first zoom-in page 170, the electronic device 900 displays a bottom-bouncing animation effect.

[0851] That is, the electronic device 900 supports multiple function dimensions and gesture dimensions of operations in the first zoom-in page 170 provided by the virtual assistant, which can meet the user's use demand and improve the user's use experience.

[0852] In step S55, in response to the click operation on the first editing control 171, the electronic device 900 displays the editing main page 174, and displays the second target content 140 and at least one first editing tool 175 on the editing main page 174, the first editing tool 175 being used for image processing on the second target content 140.

[0853] FIG. 68 is a third scenario diagram of user interaction with the virtual assistant based on the update control according to an embodiment of the present application.

[0854] As shown in FIG. 68, in one implementation, when the second target content 140 is a picture, after step S53, the method further includes steps S56-S58.

[0855] At step S56, in response to a click operation on a third target control in the update controls 141, the electronic device 900 displays a third target service recommendation 176 corresponding to the third target control in the second status window 62, and the third target service recommendation 176 is used for image processing of the second target content 140.

[0856] At step S57, the electronic device 900 displays a fifth response dialog box 177 corresponding to the third target service recommendation 176 in the second status window 62, and displays a first processed picture 178 corresponding to the second target content 140 in the fifth response dialog box 177, displays a second edit control 179 in a region adjacent to the first processed picture 178 in the fifth response dialog box 177, and displays a third edit control 180 in a region adjacent to the fifth response dialog box 177, wherein the second edit control 179 is used for linking to an edit sub-page corresponding to the third target service recommendation 176, and the third edit control 180 is used for linking to the edit main page 174.

[0857] The third edit control 180 can also be used to aggregate multiple update controls 141. The user can perform a click operation on the third edit control 180, and in response to the click operation on the third edit control 180, the electronic device 900 displays an edit menu corresponding to the third edit control 180, and displays a service recommendation corresponding to at least one update control 141 in the edit menu. The user can perform a click operation on any service recommendation, and in response to the click operation on the service recommendation, the electronic device 900 can display an edit sub-page corresponding to the service recommendation.

[0858] The second sharing control and the second save control are displayed in a region adjacent to the first processed picture 178 in the fifth response dialog box 177. The functions of the second sharing control and the second save control can refer to the foregoing embodiments, and will not be described herein.

[0859] It should be noted that the electronic device 900 can load the first processing picture 178 according to a preset proportion rule. If the electronic device 900 can predict the picture type of the first processing picture 178 based on the current scene, for example, can predict that the first processing picture 178 is an ID photo, the electronic device 900 can generate the picture proportion of the first processing picture 178 according to the predicted picture type. If the electronic device 900 cannot predict the picture type of the first processing picture 178, for example, the first processing picture 178 is a cutout, the electronic device 900 can generate the first processing picture 178 with a fixed proportion. In step S58, in response to the click operation on the first processing picture 178, the electronic device 900 displays a second zoom-in page 181, and displays the first processing picture 178 in a zoom-in state on the second zoom-in page 181, and displays a fourth editing control 182 on the second zoom-in page 181.

[0860] The second zoom-in page 181 further includes a third sharing control and a third saving control. The functions of the third sharing control and the third saving control can be referred to the foregoing embodiments, and the present application will not be repeated here.

[0861] It should be noted that in the embodiments of the present application, the saving control, the sharing control, the resetting control, the undo control and the like can be adaptively set on the page based on the virtual assistant jump to realize the corresponding functions, and the specific display form of the page based on the virtual assistant jump is not limited in the embodiments of the present application.

[0862] In an implementation manner, after the electronic device 900 displays the second zoom-in page 181 for a certain time length, the electronic device 900 can display the second zoom-in page 181 in an immersive state, that is, hide the menu bar on the page for displaying the fourth editing control 182, the third sharing control and the third saving control.

[0863] It should be noted that the function of the electronic device 900 displaying the second zoom-in page 181 in the immersive state can be adapted to the function provided by the gallery application of the electronic device 900.

[0864] In an implementation manner, in response to a double-click operation on the first processing picture 178 in the zoom-in state, or in response to a double-finger pinch-out operation on the first processing picture 178 in the zoom-in state, the electronic device 900 performs zoom processing on the first processing picture 178 in the zoom-in state.

[0865] It should be noted that the function of the electronic device 900 performing zoom processing on the first processing picture 178 in the zoom-in state based on the gesture operation is also adapted to the function provided by the gallery application of the electronic device 900.

[0866] In an implementation manner, in response to a long press operation on the first processed picture 178 in the zoom-in state, the electronic device 900 displays the first screen area 168.

[0867] In an implementation manner, in the immersive state, in response to a click operation on the first processed picture 178 in the zoom-in state, the electronic device 900 switches to the non-immersive state. And, in response to a click operation on the first processed picture 178 in the zoom-in state again, the electronic device 900 switches from the non-immersive state to the immersive state.

[0868] In an implementation manner, in response to a double-finger drag operation on the first processed picture 178 in the zoom-in state, the electronic device 900 can move the first processed picture 178.

[0869] In an implementation manner, in the case that the first processed picture 178 in the zoom-in state has text, in response to a long press operation on the first processed picture 178, the electronic device 900 can preferentially select the text in the first processed picture 178.

[0870] In an implementation manner, in the case that the first processed picture 178 in the zoom-in state does not have text, in response to a long press operation on the first processed picture 178, the electronic device 900 can display the first processed picture 178 as a draggable state.

[0871] In an implementation manner, in response to a downward sliding operation in the second zoom-in page 181, the electronic device 900 exits the second zoom-in page 181.

[0872] In an implementation manner, in response to an upward sliding operation in the second zoom-in page 181, or, in response to a leftward sliding operation in the second zoom-in page 181, or, in response to a downward sliding operation in the second zoom-in page 181, the electronic device 900 displays a bottom-bouncing animation effect.

[0873] That is, the electronic device 900 supports multiple function dimensions and gesture dimensions of operations in the second zoom-in page 181 based on the virtual assistant, which can meet the user's use demand and improve the user's use experience.

[0874] Different from the foregoing embodiment, the first editing control 171 on the first zoom-in page 170 can be linked to the editing main page 174, and the fourth editing control 182 on the second zoom-in page 181 is linked to the editing sub-page 183.

[0875] FIG. 69 is a schematic diagram of a scene in which a user interacts with a virtual assistant based on a fourth editing control according to an embodiment of the present application.

[0876] As shown in FIG. 69, in an implementation manner, after step S58, steps S59-S63 are further included.

[0877] In step S59, in response to the click operation on the fourth editing control 182, the electronic device 900 displays the editing sub-page 183, and displays the first confirmation button 184 on the editing sub-page 183.

[0878] In step S60, after the electronic device 900 performs the first editing operation on the first processed picture 178 on the editing sub-page 183, the second processed picture 185 is obtained.

[0879] It should be noted that after the electronic device 900 displays the second target content 140 in the second state window 62, the service carrier application can be jumped to, so as to provide services for the second target content 140, where the service carrier application can be, for example, a gallery application. If the service carrier application supports the corresponding service (such as the AI removal service corresponding to the editing sub-page 183) executed in the virtual assistant, the user can further edit the first processed picture 178 on the editing sub-page 183, or reset the first processed picture 178 to the initial state of the second target content 140 by clicking the reset control or the undo control. If the service carrier application does not support the corresponding service executed in the virtual assistant, the user can only perform the pre-defined operation on the editing sub-page 183, and cannot perform other operations in the gallery application, for example, when the editing sub-page 183 is a sticker page, the pre-defined operation is to add the first sticker by default, and the user cannot add other stickers on the page.

[0880] In step S61, in response to the click operation on the first confirmation button 184, the electronic device 900 displays the editing main page 174, and displays the first storage button 186 on the editing main page 174.

[0881] In step S62, after the electronic device 900 performs the second editing operation on the second processed picture 185 on the editing main page 174, the third processed picture 187 is obtained.

[0882] In step S63, in response to the click operation on the first storage button 186, the electronic device 900 stores the third processed picture 187.

[0883] When the electronic device 900 stores the third processed picture 187, the user can be prompted about the storage path of t...

Claims

1. A virtual assistant display method, characterized by, The application is applied to an electronic device, comprising: displaying a first page; waking up a virtual assistant on the first page to display a first window corresponding to the virtual assistant and at least one first control adjacent to the first window, the first control comprising a service recommendation matching the first page; updating the first page in response to a first operation on the first page to obtain a first updated page; updating the first control based on the first updated page to obtain a first updated control, the first updated control comprising the service recommendation matching the first updated page. 2.The virtual assistant display method of claim 1, characterized in that, The step of waking up the virtual assistant on the first page comprises: waking up the virtual assistant on the first page in response to a second operation on a navigation button, wherein the navigation button comprises at least one of a navigation bar, a home button in a navigation bar, and the second operation comprises at least one of a long press operation and a click operation; or, waking up the virtual assistant on the first page in response to a third operation on a power key, the third operation comprising at least one of a long press operation and a click operation; or, waking up the virtual assistant on the first page in response to an operation of dragging a first target content of the first page to the navigation button; or, displaying a first icon corresponding to the virtual assistant on a first screen area in response to an operation of dragging the first target content to the first screen area, and waking up the virtual assistant on the first page in response to an operation of dragging the first target content to the first icon. 3.The virtual assistant display method of claim 2, characterized in that, The step of waking up the virtual assistant on the first page in response to an operation of dragging the first target content of the first page to the navigation button comprises: generating a first selected area at a position where the first target content is located in response to a selection operation on the first target content; generating a first moving area for copying and moving the first target content in response to a long press operation on the first selected area; displaying a transition window at a first position corresponding to the navigation button in response to an operation of dragging the first moving area to the navigation button, the transition window being used to prompt a user that the first position can wake up the virtual assistant; waking up the virtual assistant on the first page in a case where the first moving area reaches a first preset area, wherein the first preset area is an area adjacent to the transition window, and the first preset area is used to avoid a screen hot area. 4.The virtual assistant display method of claim 3, characterized in that, The step of waking up the virtual assistant on the first page in a case where the first moving area reaches a first preset area further comprises: generating a second control in a region adjacent to the first window in response to an operation of dragging the first moving area to the region adjacent to the first window, the second control being used to cache the first target content. 5.The virtual assistant display method of claim 4, characterized in that, The method further comprises, after the step of generating the second control in the region adjacent to the first window: updating the first control based on the first target content in the second control to obtain a second updated control, the second updated control comprising the service recommendation matching the first target content. 6.The virtual assistant display method of claim 2, wherein, The display of the first window corresponding to the virtual assistant comprises: In the case of waking up the virtual assistant in response to the second operation on the navigation button, or in the case of waking up the virtual assistant in response to the operation of dragging the first target content of the first page to the navigation button, the first window with a first input state is displayed, and the first input state is a state of not currently receiving voice input. In the case of waking up the virtual assistant in response to the third operation on the power key, the first window with a second input state is displayed, and the second input state is a state of currently receiving voice input. 7.The virtual assistant display method of claim 2, wherein, The display of the first window corresponding to the virtual assistant comprises: In the case that the first page displays the navigation bar, the first window is displayed in the adjacent area of the navigation bar to avoid the navigation bar.

8. The virtual assistant display method according to any one of claims 2 to 7, characterized by, Further comprising: In the case of displaying the first window, a first interaction area is displayed in the first window, and the first interaction area is used to receive the input operation of the user. In response to the click operation on the first interaction area, an input method panel is displayed, and the input method panel covers the first window to make the first window invisible. In response to the operation of exiting the input method panel, the input method panel is hidden, and the first window is displayed.

9. The virtual assistant display method of any one of claims 2-7, wherein: The first window comprises a first state window, a second state window, and a third state window. The first state window comprises a first interaction area, and the first interaction area is used to receive the input operation of the user. The second state window comprises a first sub-window and a second sub-window, the first sub-window comprises the first interaction area and a first panel, the first panel is displayed in a part of the display screen of the electronic device, the first panel is used to accommodate the interaction content between the virtual assistant and the user, the second sub-window comprises the first interaction area and a second panel, the second panel is displayed in a part of the display screen of the electronic device, and the panel height of the second panel is higher than the panel height of the first panel, and the second panel is used to accommodate the interaction content between the virtual assistant and the user. The third state window comprises the first interaction area and a third panel, the third panel is displayed in a full-screen area corresponding to at least one of the display screens, and the third panel is used to accommodate the interaction content between the virtual assistant and the user. 10.The virtual assistant display method of claim 9, characterized in that, The display of the first window corresponding to the virtual assistant and at least one first control adjacent to the first window comprises: The first state window is displayed, and the first control is displayed in the adjacent area of the outer periphery of the first state window. Or, the first sub-window is displayed, and the first control is displayed in the first panel, and the first control is displayed in a peripheral adjacent area of the first interaction area. 11.The virtual assistant display method of claim 10, characterized in that, After the first window corresponding to the virtual assistant and the at least one first control adjacent to the first window are displayed, the method further includes: In response to an input operation on the first status window, the first sub-window is displayed, and interaction content corresponding to the input operation is displayed in the first panel; In the first panel, if an area of a display region corresponding to the interaction content is greater than a first area threshold, the second sub-window is displayed, and the interaction content is displayed in the second panel. 12.The virtual assistant display method of claim 10, characterized in that, After the first window corresponding to the virtual assistant and the at least one first control adjacent to the first window are displayed, the method further includes: In response to a fourth operation on the first control, the first sub-window is displayed, and interaction content corresponding to the first control is displayed in the first panel, the fourth operation including at least one of a click operation and a drag operation; In the first panel, if an area of a display region corresponding to the interaction content is greater than a first area threshold, the second sub-window is displayed, and the interaction content is displayed in the second panel. 13.The virtual assistant display method of claim 11 or 12, characterized in that, The method further includes: In a case where the first status window is displayed, in response to an upward sliding operation on a navigation bar in the first status window, at least one of the first sub-window, the second sub-window, and the third status window is displayed; In a case where the first sub-window is displayed, in response to an upward sliding operation on a navigation bar in the first sub-window, at least one of the second sub-window and the third status window is displayed; In a case where the second sub-window is displayed, in response to an upward sliding operation on a navigation bar in the second sub-window, the third status window is displayed. 14.The virtual assistant display method of claim 11 or 12, wherein, The method further includes: In a case where the third status window is displayed, in response to a downward sliding operation on a top area of the display screen, at least one of the first status window, the first sub-window, and the second sub-window is displayed, or the third status window is minimized to the navigation button; In a case where the second sub-window is displayed, in response to a downward sliding operation on a navigation bar in the second sub-window, at least one of the first status window and the first sub-window is displayed, or the second sub-window is minimized to the navigation button; In a case where the first sub-window is displayed, in response to a downward sliding operation on a navigation bar in the first sub-window, the first status window is displayed, or the first sub-window is minimized to the navigation button. 15.The virtual assistant display method of claim 10, wherein, After the first status window is displayed, the method further includes: In response to a long-pressing and dragging operation on the navigation bar in the first status window, the first status window is displayed as a corresponding movement status window of the first status window, the movement status window including the first interaction area and a fourth panel, the fourth panel being used to accommodate the first control; In response to an operation of dragging the moving state window to a target position, the moving state window is displayed at the target position. 16.The virtual assistant display method of claim 11 or 12, wherein, The displaying of the second sub-window further includes: In response to an operation of long pressing and dragging a navigation bar in the second sub-window, a panel height of the second panel is reduced to a panel height of the first panel, so that the second sub-window is transformed into the first sub-window. In response to an operation of dragging the first sub-window to a target position, the first sub-window is displayed at the target position. 17.The virtual assistant display method of claim 13, wherein, The displaying of the third state window further includes: In response to an operation of long pressing and dragging a top area of the display screen, the third state window is displayed as the second state window. In response to an operation of dragging the second state window to a target position, the second state window is displayed at the target position. 18.The virtual assistant display method of claim 13, wherein, The displaying of the third state window further includes: In a case where the electronic device is a foldable electronic device, the foldable electronic device is in an unfolded state, and the third state window is displayed in a full-screen area corresponding to the entire display screen, in response to an operation of long pressing and dragging along a top edge of a top area of the display screen, the third state window is displayed in a full-screen area corresponding to a part of the display screen based on a split-screen mode, and the first page is displayed in a full-screen area corresponding to another part of the display screen.

19. The virtual assistant display method according to claim 11 or 12, characterized by, Further includes: In a case where the first page is updated in response to a first operation on the first page, if the first state window or the second state window is displayed on the first page, the display states of the first state window and the second state window remain unchanged.

20. The virtual assistant display method according to any one of claims 1-19, wherein, The updating of the first page in response to the first operation on the first page includes: In response to the first operation on the first page, the first page is updated, and the first control is hidden; In response to stopping the first operation, the first updated page is obtained; The first operation includes at least one of an up-swipe operation, a down-swipe operation, an operation of returning to a parent page, an operation of entering a child page, and an operation of jumping to a second page and then returning to the first page.

21. The virtual assistant display method of claim 20, wherein, The updating of the first control based on the first updated page includes: Within a first preset time length after the first operation is stopped, the first control is updated based on the first updated page; In a case where the first preset time length is reached, the first updated control is displayed.

22. The virtual assistant display method according to any one of claims 1 to 19, characterized by, The updating of the first page in response to the first operation on the first page includes: In response to the first operation on the first page, a first focus is set on the first page to focus on the first page, so as to control the first operation to match the first page; In a case where the first operation matches the first page, the first page is controlled to be updated according to the first operation to obtain the first updated page.

23. The virtual assistant display method of claim 22, wherein, The response to the first operation on the first page, setting a first focus point on the first page to focus on the first page, further comprises: displaying the first window in a defocused state; in response to a click operation on the first window, moving the first focus point to the first window to display the first window from the defocused state to a focused state; focusing on the first window when the first window is displayed in the focused state, and then controlling a fifth operation to match the first window; in the case that the fifth operation matches the first window, controlling the first window to respond to the fifth operation, the fifth operation including at least one of an upslide operation, a downslide operation, and an input operation.

24. The virtual assistant display method of claim 23, wherein, The first update control is obtained by updating the first control based on the first updated page, comprising: if the first focus point stays in the first split screen page corresponding to the first updated page when the electronic device is a foldable electronic device and the foldable electronic device is in an unfolded state and the first updated page is displayed based on a split screen mode, updating the first control based on the first split screen page to obtain the first update control, the first update control including the service recommendation matching the first split screen page; if the first focus point stays in the second split screen page corresponding to the first updated page, updating the first control based on the second split screen page to obtain the first update control, the first update control including the service recommendation matching the second split screen page. 25.The virtual assistant display method of claim 23, wherein, The first update control is obtained by updating the first control based on the first updated page, comprising: if the first focus point stays in the first split screen page corresponding to the first updated page when the electronic device is a foldable electronic device and the foldable electronic device is in an unfolded state and the first updated page is displayed based on a split screen mode, updating the first control based on the first split screen page to obtain the first update control, the first update control including the service recommendation matching the first split screen page; 26. The virtual assistant display method according to claim 11 or 12, characterized by, After the first update control is obtained by updating the first control based on the first updated page, further comprising: if the virtual assistant is woken up for a second preset time period, the first window is minimized to the navigation button, and the interaction content is cached as historical content.

27. The virtual assistant display method of claim 26, wherein, After the first window is minimized to the navigation button and the interaction content is cached as historical content, further comprising: waking up the virtual assistant again on the first page to display the first window corresponding to the virtual assistant, and displaying the historical content in the first window.

28. The virtual assistant display method of any one of claims 1-27, wherein, After the first update control is obtained by updating the first control based on the first updated page, further comprising: in response to an operation of sliding up from the bottom of the display screen of the electronic device, ending the session of the virtual assistant.

29. The virtual assistant display method of claim 9, wherein, After the virtual assistant is woken up on the first page to display the first window corresponding to the virtual assistant, further comprising: in response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window, wherein each of the third update controls is configured to provide at least one service recommendation matching the second target content. 30.The virtual assistant display method of claim 29, wherein, in response to a box selection operation on the target content, generating a first box selection area at a position corresponding to the target content, so that the target content in the first box selection area is executable for a drag operation, wherein the target content comprises the first target content and the second target content. 31.The virtual assistant display method of claim 29, wherein, in a case that the first page is a home screen page and the first status window is displayed, displaying a first accessory area in a region adjacent to the first status window, wherein the first accessory area comprises at least one accessory entry configured to provide the second target content.

32. The virtual assistant display method of claim 31, wherein, the response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window comprises: in response to a click operation on a target accessory entry in the accessory entries, displaying an accessory page corresponding to the target accessory entry, and displaying the second target content in the accessory page; in response to a selection operation on the second target content, sending the second target content to the virtual assistant to display the second status window on the first page, and display the second target content in the second status window, and display the third update control in the second status window.

33. The virtual assistant display method of claim 30, wherein, the response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window comprises: in response to a drag operation of the second target content to the first status window, changing from displaying the first status window to displaying the second status window, and displaying the second target content in the second status window, and display the third update control in the second status window.

34. The virtual assistant display method of claim 29, wherein, the response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window further comprises: In a case where the second target content is a picture, in response to a click operation on a first target control in the third update control, a first target service recommendation corresponding to the first target control is displayed in the first window, and the first target service recommendation is uploaded to a first server, the first server including a visual question and answer (VQA) server; In a process in which the first target service recommendation is uploaded to the first server, a first response dialog box corresponding to the first target service recommendation is displayed in the first window, and first waiting information is displayed in the first response dialog box, the first waiting information being used to prompt a user that the electronic device is in a process of obtaining a first VQA result corresponding to the first target service recommendation; After the first VQA result issued by the first server for the first target service recommendation is obtained, the first VQA result is displayed in the first response dialog box.

35. The virtual assistant display method of claim 34, wherein, After the first target service recommendation is uploaded to the first server, the first response dialog box corresponding to the first target service recommendation is displayed in the first window, and the first waiting information is displayed in the first response dialog box, the method further includes: When the first VQA result issued by the first server for the first target service recommendation is not obtained, first error information is displayed in the first response dialog box, and a first error button is displayed in a region adjacent to the first target service recommendation, the first error information being used to prompt the user that the first server is abnormal; In response to a click operation on the first error button, the first error button is stopped from being displayed, and the first waiting information is displayed in the first response dialog box; After the first VQA result issued by the first server for the first target service recommendation is obtained, the first VQA result is displayed in the first response dialog box.

36. The virtual assistant display method of claim 35, wherein, After the first target service recommendation is uploaded to the first server, the first response dialog box corresponding to the first target service recommendation is displayed in the first window, and the first waiting information is displayed in the first response dialog box, the method further includes: When the first VQA result issued by the first server for the first target service recommendation is not obtained, first error information is displayed in the first response dialog box, and a first error button is displayed in a region adjacent to the first target service recommendation, the first error information being used to prompt the user that the first server is abnormal; In response to a sixth operation of inputting a first instruction into the first window, the first error button is stopped from being displayed, a second response dialog box corresponding to the first instruction is displayed in the first window, and second waiting information is displayed in the second response dialog box, the second waiting information being used to prompt the user that the electronic device is in a process of obtaining a first response result corresponding to the first instruction; After the first response result is obtained, the first response result is displayed in the second response dialog box.

37. The virtual assistant display method of claim 29, wherein, The method further includes: In response to a click operation on a second target control in the third update control, displaying a second target service recommendation corresponding to the second target control in the first window, and uploading the second target service recommendation to a second server, the second server including a document question and answer (DQA) server; In the process of uploading the second target service recommendation to the second server, displaying a third response dialog box corresponding to the second target service recommendation in the first window, and displaying third waiting information in the third response dialog box, the third waiting information being used to prompt that the electronic device is in a process of obtaining a first DQA result corresponding to the second target service recommendation; After obtaining the first DQA result issued by the second server for the second target service recommendation, displaying the first DQA result in the third response dialog box.

38. The virtual assistant display method of claim 37, wherein, The method further includes: In response to a click operation on the second error button, stopping displaying the second error button, and displaying the third waiting information in the third response dialog box; After obtaining the first DQA result issued by the second server for the second target service recommendation, displaying the first DQA result in the third response dialog box. The method further includes:

39. The virtual assistant display method of claim 37, wherein, In response to a click operation on the second error button, stopping displaying the second error button, and displaying the third waiting information in the third response dialog box; After obtaining the first DQA result issued by the second server for the second target service recommendation, displaying the first DQA result in the third response dialog box. The method further includes: In response to a click operation on the second error button, stopping displaying the second error button, and displaying the third waiting information in the third response dialog box; In response to a seventh operation of inputting a second instruction into the first window, stop displaying the second error reporting button, display a fourth response dialog box corresponding to the second instruction in the first window, and display fourth waiting information in the fourth response dialog box, the fourth waiting information being used to prompt the user that the electronic device is in a process of obtaining a second response result corresponding to the second instruction; After obtaining the second response result, display the second response result in the fourth response dialog box.

40. The virtual assistant display method of claim 29, wherein, The response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window further includes: In a case where the second target content is a picture, in response to a click operation on the picture, a first zoom-in page is displayed, the picture in a zoom-in state is displayed on the first zoom-in page, and a first edit control is displayed on the first zoom-in page; In response to a click operation on the first edit control, an edit main page is displayed, the picture is displayed on the edit main page, and at least one first edit tool is displayed on the edit main page, the first edit tool being used for image processing of the picture.

41. The virtual assistant display method of claim 40, wherein, The response to the click operation on the picture, the display of the first zoom-in page, the display of the picture in the zoom-in state on the first zoom-in page, and the display of the first edit control on the first zoom-in page further includes: In response to a double-click operation on the picture in the zoom-in state, or in response to a double-finger pinch operation on the picture in the zoom-in state, the picture in the zoom-in state is zoomed out.

42. The virtual assistant display method of claim 29, wherein, The response to sending the second target content on the first page to the virtual assistant, displaying the second target content in the first window, and displaying at least one third update control associated with the second target content in the first window further includes: In response to a click operation on a third target control in the third update control, a third target service recommendation corresponding to the third target control is displayed in the first window, the third target service recommendation being used for image processing of the picture; In the first window, a fifth response dialog box corresponding to the third service recommendation is displayed, a first processed picture corresponding to the picture is displayed in the fifth response dialog box, a second edit control is displayed in a region adjacent to the first processed picture in the fifth response dialog box, and a third edit control is displayed in a region adjacent to the fifth response dialog box, the second edit control being used to link to an edit sub-page corresponding to the third service recommendation, and the third edit control being used to link to an edit main page, the edit main page including at least one first edit tool, the first edit tool being used for image processing of the first processed picture.

43. The virtual assistant display method of claim 42, wherein, The method of displaying a fifth response dialog box corresponding to the third service recommendation in the first window, displaying a first processed image corresponding to the image in the fifth response dialog box, displaying a second editing control in the area adjacent to the first processed image in the fifth response dialog box, and displaying a third editing control in the area adjacent to the fifth response dialog box, further includes: In response to a click operation on the first processed image, a second zoom-in page is displayed, and the first processed image in zoomed-in state is displayed on the second zoom-in page, as well as a fourth editing control is displayed on the second zoom-in page; In response to a click on the fourth editing control, the editing subpage is displayed, and a first confirmation button is shown on the editing subpage; After performing a first editing operation on the first processed image on the editing subpage, a second processed image is obtained; In response to a click on the first confirmation button, the main editing page is displayed, and a first save button is displayed on the main editing page; After performing a second editing operation on the second processed image on the main editing page, a third processed image is obtained; In response to a click on the first storage button, the third processed image is stored.

44. The virtual assistant display method of claim 43, wherein, After responding to a click operation on the first processed image, displaying a second zoom-in page, and displaying the first processed image in a zoomed-in state on the second zoom-in page, and displaying a fourth editing control on the second zoom-in page, the method further includes: In response to a double-click operation on the first processed image in the zoomed-in state, or in response to a two-finger zoom-out operation on the first processed image in the zoomed-in state, the first processed image in the zoomed-in state is scaled.

45. The virtual assistant display method of claim 42, wherein, The method of displaying a fifth response dialog box corresponding to the third service recommendation in the first window, displaying a first processed image corresponding to the image in the fifth response dialog box, displaying a second editing control in the area adjacent to the first processed image in the fifth response dialog box, and displaying a third editing control in the area adjacent to the fifth response dialog box, further includes: In response to a click operation on the second editing control, the editing subpage is displayed, and a first confirmation button is displayed on the editing subpage; After performing a first editing operation on the first processed image on the editing subpage, a second processed image is obtained; In response to a click on the first confirmation button, the main editing page is displayed, and a first save button is displayed on the main editing page; After performing a second editing operation on the second processed image on the main editing page, a third processed image is obtained; In response to a click on the first storage button, the third processed image is stored.

46. The virtual assistant display method of claim 42, wherein, The third service recommendation corresponding to the fifth response dialog box is displayed in the first window, a first processed picture corresponding to the picture is displayed in the fifth response dialog box, a second editing control is displayed in a region adjacent to the first processed picture in the fifth response dialog box, and a third editing control is displayed in a region adjacent to the fifth response dialog box, and the method further comprises: In response to a click operation on the third editing control, the editing main page is displayed, and a first storage button is displayed on the editing main page; After a third editing operation is performed on the first processed picture on the editing main page, a fourth processed picture is obtained; In response to a click operation on the first storage button, the fourth processed picture is stored.

47. An electronic device, comprising: The method comprises: A processor and a memory, the memory storing program instructions that, when executed by the processor, cause the electronic device to perform the virtual assistant display method of any one of claims 1-46.

48. A computer-readable storage medium, comprising: The method comprises: Computer instructions that, when executed on an electronic device, cause the electronic device to perform the virtual assistant display method of any one of claims 1-46.

Images