Intelligent task guiding method, device and system based on digital assistant

Abstract

The invention provides an intelligent guiding method, device and system based on a digital assistant, which are mainly applied to electronic equipment comprising a display screen and carrying the digital assistant, and the method comprises the following steps: the electronic equipment receives a user instruction and generates an automatic task; and based on the selected automated task execution mode and automated task step execution mode, executing the automated task and obtaining an execution result. By using the method, the user can be effectively helped to understand the execution steps and the execution process of the whole automatic task, the user can be helped to quickly learn corresponding task processing, meanwhile, the corresponding task can be automatically completed based on the willingness of the user on the basis of a digital assistant and a task island mode, the user experience can be improved, and the user experience is improved. And two hands can be liberated and the task processing efficiency can be improved only by inputting the task step corresponding to the voice indication.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and system for intelligent task guidance based on a digital assistant. Background Art

[0002] With the rapid development of society and technological advancements, smartphone and smart car manufacturers are constantly updating and upgrading the operating systems installed in these smart terminals, developing more and more features to meet users' growing needs. However, as the number of functions provided by terminal operating systems increases, some software features, beyond those commonly used by users, may be difficult for users to perceive and use after development. As a result, even some convenient features that can replace tedious user manuals may not be recognized and quickly utilized. As a result, the developed software functions fail to meet the target needs and users lack a good user experience.

[0003] To address this issue, existing smart terminals are equipped with virtual assistants, which help users perform simple functions by giving commands to the virtual assistants. However, the execution process of virtual assistants is a black box, and the final result is provided. If the user is dissatisfied with the current execution result of the virtual assistant, the user still does not know how to manually execute the function to achieve the desired result, which results in a poor user experience.

[0004] Based on this, how to help users quickly learn the task operation process and obtain task results based on user intentions by displaying the specific steps of task execution and single-step interface execution guidance is an urgent problem that needs to be solved. Summary of the Invention

[0005] The present application proposes an intelligent task guidance method based on a digital assistant. The method generates an automated task by supporting the reception of user instructions, and executes the automated task and obtains the execution result based on the user's execution mode of the automated task and the task steps in the automated task. The execution mode of the automated task includes a one-key generation mode and a task island mode, and the execution mode of the task step includes automatic execution and manual execution. The user can use the task island mode to execute the automated task according to the needs of the actual usage scenario, and indicate the execution of the task step before the task step of the automated task is executed. With the help of a digital assistant and the task island mode, this method can not only obtain the user's intention and instructions before the execution of each task step, but also execute the automated task in full accordance with the user's intention, which can effectively improve the user experience.

[0006] In the first aspect, a method for intelligent task guidance based on a digital assistant is proposed. This method is primarily applicable to electronic devices that include a display screen and a digital assistant. The method includes: the electronic device receiving a user instruction and generating an automated task; executing the automated task based on a selected automated task execution mode and an automated task step execution mode, and obtaining an execution result. The method then executes the automated task according to the user's wishes by obtaining the user's selection result.

[0007] In a possible implementation of the first aspect, the automated task execution mode includes a one-key generation mode and a task island mode.

[0008] In a possible implementation of the first aspect, the one-key generation mode is a mode in which the digital assistant black box executes the automation task; the task island mode is a mode in which the automation task is executed in a single step.

[0009] In this solution, automated task execution modes include one-click generation mode and task island mode. The task island mode is the core innovation of this solution. In task island mode, users can watch the entire execution process of the automated task. The automated task is executed step by step, and the user's consent is asked for at each execution step. This solution allows users to participate in the entire process and understand the entire automated task execution process and execution path. Even if the user subsequently leaves the digital assistant, the user can still complete the task step by step by themselves.

[0010] In a possible implementation of the first aspect, in the task island mode, the display screen displays the application interface corresponding to the current step and displays the task island card, and the task island card displays at least one of the execution path of the current step of the automation task, the task island icon, the execution progress of the automation task, the operation control, and the operation object.

[0011] In a possible implementation of the first aspect, the task island mode executes multiple steps of the automated task in a single step, and after each step is executed, the application interface corresponding to the current step remains displayed within a preset time, and the task island card is displayed.

[0012] In a possible implementation of the first aspect, in the task island mode, during the execution of the steps of the automated task, the task island card is displayed in a hidden display manner, and the hidden display manner includes: not displaying in the interface, hiding along the edge, and displaying in the top status bar after performing at least one of shrinking and deforming.

[0013] In this solution, in the task island mode, in the display interface, the task island card can be displayed in a hidden display manner, that is, it can be completely hidden in the interface, or hidden against the edge, or displayed in the top status bar after performing at least one of shrinking and deforming. The task island can be displayed in a hidden display manner not only to ensure the full-screen display of the display interface, but also to be displayed or hidden according to user needs, which can effectively improve the user experience.

[0014] In a possible implementation of the first aspect, when the task island card is displayed, the display screen displays at least one control, and the at least one control is used to indicate the execution method of the next step of the automated task, the execution method of the next step includes manual and automatic, and the at least one control includes a manual execution control.

[0015] In this solution, during the execution of a single step of the automated task, there will be a selection box before each step is executed. The selection box contains at least a control for indicating manual execution. The user can operate the control to manually execute the task step according to his or her own wishes, which is conducive to improving the user experience and can help to refine the user's wishes and apply them to each task step.

[0016] In a possible implementation of the first aspect, in the task island mode, a digital assistant icon is displayed in the application interface corresponding to the current step, and sliding up the digital assistant icon displays the dialogue interface and dialogue content corresponding to the digital assistant.

[0017] In this solution, users can view the dialog box and conversation content by swiping up the digital assistant, which helps users make the next judgment and execution based on the conversation content.

[0018] In a possible implementation of the first aspect, in the process of executing the task steps of the automated task using the task island mode, the digital assistant executes the task steps based on the user's intention.

[0019] In a possible implementation of the first aspect, the user intention is obtained by the digital assistant based on at least one of active inquiry, receiving active user input instructions, and responding according to a predefined response after timeout.

[0020] In a possible implementation manner of the first aspect, the receiving of the user's active input instruction occurs before the execution of the task step.

[0021] On the second aspect, an electronic device including a display screen and equipped with a digital assistant is proposed, which is used to perform intelligent task guidance based on the digital assistant, including: a receiving module for receiving user instructions and generating automated tasks; a processing module for executing the automated tasks based on the selected automated task execution mode and automated task step execution mode and obtaining execution results.

[0022] In a possible implementation manner of the second aspect, the automated task execution mode includes a one-key generation mode and a task island mode.

[0023] In a possible implementation of the second aspect, the one-key generation mode is a mode in which the digital assistant black box executes the automation task; the task island mode is a mode in which the automation task is executed in a single step.

[0024] In a possible implementation of the second aspect, in the task island mode, the display screen displays the application interface corresponding to the current step and displays the task island card, and the task island card displays at least one of the execution path of the current step of the automation task, the task island icon, the execution progress of the automation task, the operation control, and the operation object.

[0025] In a possible implementation of the second aspect, the task island mode executes multiple steps of the automated task in a single step, and after each step is executed, the application interface corresponding to the current step remains displayed within a preset time, and the task island card is displayed.

[0026] In a possible implementation of the second aspect, in the task island mode, during the execution of the steps of the automated task, the task island card is displayed in a hidden display manner, and the hidden display manner includes: not displaying in the interface, hiding along the edge, and displaying in the top status bar after performing at least one of shrinking and deforming.

[0027] In a possible implementation of the second aspect, when the task island card is displayed, the display screen displays at least one control, and the at least one control is used to indicate the execution method of the next step of the automated task, the execution method of the next step includes manual and automatic, and the at least one control includes a manual execution control.

[0028] In a possible implementation of the second aspect, in the task island mode, a digital assistant icon is displayed in the application interface corresponding to the current step, and sliding up the digital assistant icon displays the dialogue interface and dialogue content corresponding to the digital assistant.

[0029] In a possible implementation of the second aspect, in the process of executing the task steps of the automated task using the task island mode, the digital assistant executes the task steps based on the user's intention.

[0030] In a possible implementation of the second aspect, the user intention is obtained by the digital assistant based on at least one of active inquiry, receiving active input instructions from the user, and responding according to a predefined response after timeout.

[0031] In a possible implementation manner of the second aspect, the receiving of the user's active input instruction occurs before the execution of the task step.

[0032] In a third aspect, an electronic device with a display interface is proposed, wherein the electronic device includes a processor, the processor is coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the electronic device executes a method as described in any one of the first or second aspects above.

[0033] In a fourth aspect, a computer-readable storage medium is proposed, characterized in that it includes computer instructions, and when the computer instructions are executed on a computer system, the computer system implements the method described in any one of the first aspect or the second aspect above.

[0034] In a fifth aspect, a computer program product is proposed, which includes instructions, and is characterized in that when the instructions are executed, the computer implements the method as described in any one of the first aspect or the second aspect above.

[0035] The beneficial effects of the second to fifth aspects mentioned above can be referred to the introduction of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of an interface display for performing tasks using a digital assistant provided in this embodiment;

[0038] Figure 3 (a) Figure 3 (b) is a process step display interface for creating a task island using the task island mode provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of an interface display of task execution steps supporting user interaction provided in an embodiment of the present application;

[0040] Figure 5A schematic diagram of a digital opponent dialog box display in a task island mode provided in an embodiment of the present application;

[0041] Figure 6 An embodiment of the present application provides an electronic device including a display screen and equipped with a digital assistant. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of this application, rather than all embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0043] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchangeable where appropriate so that the embodiments can be implemented in a sequence other than that illustrated or described in this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0044] The division of units in this application is a logical division. In actual application, there may be other division methods. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. Moreover, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed into multiple circuit units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

[0045] To facilitate understanding, some technical terms involved in the embodiments of this application are first introduced below.

[0046] (1) Smart Cockpit

[0047] The intelligent cockpit integrates advanced information technology and automated systems within the vehicle to upgrade and optimize the in-vehicle environment while simultaneously connecting with the external environment to provide a higher level of driving experience and riding comfort. This concept encompasses multiple aspects, including driver and passenger entertainment, information acquisition, vehicle control, and interaction with the external world.

[0048] (2) Digital Assistant

[0049] A digital assistant is an advanced computer program that typically runs on the internet and can simulate human conversations with users. It can leverage advanced artificial intelligence (AI), natural language processing, natural language understanding, and machine learning technologies for self-learning, providing users with a personalized conversational experience.

[0050] (3) Automated tasks

[0051] In this application, an automated task includes at least one step, and each step is associated with at least one functional interface, which implements the function corresponding to the step through the functional interface. For example, the functional interfaces required for automatic phone calls include the dialing interface and the contact interface. The contact interface is called to find a specific contact, and then the dialing interface is used to dial the contact.

[0052] (4) Functional interface

[0053] In this application, a functional interface is an existing interface in the operating system, usually corresponding to a menu item in the application interface, which can be used to enter a certain interface and perform a certain function. For example, in a gallery application, when viewing a picture, there are usually menu items such as Edit, Send, Delete, and Favorite. These menu items are all considered functional interfaces.

[0054] This application proposes a method for displaying an interface during the execution of an automated task. This method meets the user's need for interaction with the digital assistant during the execution of the task by displaying the progress of the automated task on the interface. The user can intervene or instruct the execution of the task through interaction with the digital assistant. This process does not pose a risk of exposing user privacy and can effectively improve the user experience.

[0055] See also Figure 1 , Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1As shown, the application scenario includes an electronic device 001 and a user 002. This scenario is mainly for demonstrating the interaction between the electronic device 001 and the user 002. In this embodiment, the electronic device 001 is a smart terminal electronic device such as a smart phone or a smart car: it has a built-in operating system, and the operating system supports the installation of system applications and third-party applications. The operating system has AI capabilities, such as a built-in digital assistant, AI applications, etc. The electronic device 001 can use the AI ​​capabilities to call the installed applications or the interfaces provided by the applications, and the AI ​​capabilities can interpret the user's typed content or voice information to obtain the user's intention information, and generate and execute automated tasks based on the user's intention information. The automated tasks may include one or more steps; the user 002 interacts with the digital assistant through voice, text typing, touch, etc., so that the digital assistant can receive and recognize the user's intention and instructions, and then execute the corresponding task process based on the intention and instructions, thereby achieving quick operation, changeable process, and better completion of the corresponding task according to the user's intention.

[0056] exist Figure 1 In the example, user 002 inputs the voice message "How do I make an ID photo?", and electronic device 001 receives the corresponding voice signal and analyzes the voice signal to obtain the user's intention, and generates and executes the corresponding automated task based on the user's 002 intention.

[0057] In a possible implementation of this embodiment, a display interface is built into the smart cockpit, and user 002 interacts with the digital assistant through the display interface.

[0058] In another embodiment, electronic device 001 may also be a terminal device with a display interface and a digital assistant, such as a mobile phone, tablet, computer, watch, robot, etc. User 001 can interact with the digital assistant through the display interface. The digital assistant has the capability of an AI big model and can implement the big model capability by calling at least one of a cloud-based big model and a built-in big model. The electronic device can call the digital assistant to provide services in any interface. For easy viewing and setting, the execution progress of the task will also be displayed on the display interface, allowing user 001 to view, intervene, and set in real time.

[0059] In this embodiment, the electronic device receives input information and creates automated tasks based on its understanding of the input information. Specifically, the electronic device can wake up the digital assistant to provide services in any display interface. This is because the digital assistant is usually resident in the background and in standby mode, so it can detect and receive user input in real time. Some digital assistants can also be activated through operations such as voice wake-up. After activation, the digital assistant can also try to detect and receive user input to better provide services to the user.

[0060] In one possible implementation of this embodiment, the digital assistant has the ability to receive user input. The user can provide input information through at least one of voice input and text typing. For different input messages received, the digital assistant can perform semantic understanding and intent recognition on the input information based on the large model capability, and finally generate a series of automated tasks based on semantics, intent and other information. The automated task usually includes multiple steps, and the multiple steps are usually completed through multiple computer instructions. The digital assistant can generate computer instructions and execute each step step by step until all steps are finally completed.

[0061] In smart terminal applications, digital assistants can be displayed on the front-end interface or not, but they are always present in the background and can be activated by voice or monitor text input in real time. In smart car applications, voice input may be more common, especially when the user is driving, where voice input is safer and more convenient. Of course, for users waiting at red lights, text typing or touchscreen interaction is also possible. For mobile phones, both voice and text input methods exist.

[0062] When the user expects to achieve a certain intention, he can wake up the digital assistant and give instructions to the digital assistant. He can choose to implement this function completely through the digital assistant, or he can express the user's intention through voice or text while the digital assistant is performing the task. The digital assistant receives the voice or text instructions, recognizes the user's intention, and executes according to the user's intention.

[0063] See also Figure 2 , Figure 2 A schematic diagram of an interface display for using a digital assistant to perform tasks provided in this embodiment. Figure 2 In this example, asking about making ID photos, after the electronic device receives the user's input command "Assistant, how do I make an ID photo on my phone?", it analyzes the user's intention and finds that the user's intention is "make an ID photo". Based on the user's intention, the function interface is matched to Gallery->Create->Export ID photos. The implementation process of the task is divided into four steps based on the intention, and brief information of the steps is displayed on the assistant's dialogue interface. The decomposition of each step and the summary and display of brief information can be achieved through the assistant's AI capabilities, so I will not go into details here.

[0064] like Figure 2As shown in the figure, the electronic device first receives the user's input command "Assistant, how do I make an ID photo on my phone?" and converts it into text and displays it in a dialog box. The digital assistant uses the background AI capabilities to match functions, extract information, etc., and finally displays the following reply information in the corresponding digital assistant dialog box: "You can use the self-made ID photo function in the Gallery App to make an ID photo. The specific steps are as follows: 1. Select a portrait photo; 2. Cut out the portrait; 3. Set the background and size; 4. Export the ID photo. Do you need me to help you make it?" After the user receives the digital assistant's request for help, he or she gives a "yes" reply, and the digital assistant Based on the user's request for "needs", the digital assistant once again throws the option to the user, "I can generate an ID photo for you with one click, or I can start the task island to help you generate a more satisfactory photo. Which method do you want?" and displays the option controls "One-click generation" and "Task Island" on the screen for users to choose. It is worth noting that the "one-click generation option" corresponds to the process of the digital assistant's black box implementation. The user is not visible to the execution process of the automated task, but the user can get the result directly; the "Task Island" option supports the digital assistant to display the task execution process and supports users to interact with it in real time and manually intervene in the steps in the execution process. The latter is more inclined to combine the process generated by the user's wishes. The former is more convenient and does not require too much intervention and participation from the user; the latter is more humane, allowing users to understand the execution process of the entire process, and during the execution process, if the user has the need to manually complete the task steps, the steps can be manually completed in full accordance with the user's needs. The combination of these two options can meet all user needs in different scenarios. When users have few requirements for their ID photos, they can choose the "One-click Generation" option for ease and convenience. When users need an ID photo with a fixed size, background, or other requirements, they can choose the "Task Island" option to fully participate in the ID creation process and adjust the execution steps based on actual needs to obtain an ID photo that meets their needs. The combination of these two methods can effectively improve all user needs in various scenarios, thereby effectively enhancing the user experience.

[0065] In one possible implementation of this embodiment, the electronic device receives a user input instruction and performs intent analysis based on the user input instruction to determine at least one task step to implement the intent, wherein each task step is associated with at least one functional interface provided by the operating system. After receiving the user input instruction, the digital assistant performs actions such as semantic understanding and intent recognition on the user instruction. These actions are basically consistent with the processing of input instructions by existing digital assistants. After semantic understanding and intent recognition, the digital assistant determines an automated task based on the understood semantics and intent. In the process of executing an automated task, the digital assistant may call multiple functional interfaces in the execution step, where each functional interface is used to implement a specific function. The digital assistant selects a functional interface based on the functional description of the functional interface when creating the automated task, or searches for a matching interface in the functional interfaces already in the operating system according to the different tasks to be performed, and then implements the function of each step by executing different functional interfaces in different orders. Specifically, the relationship between the operating system, the functional interface, and the digital assistant is not specifically limited here, as long as it can meet the requirements of the digital assistant matching the corresponding functional interface according to the specific function or calling the functional interface to implement the corresponding function.

[0066] In one possible implementation of this embodiment, after determining the corresponding automated task based on the user's instructions, the digital assistant supports the user in either execution mode: one-click generation and task island. The one-click generation mode is a fast and convenient black-box execution mode, while the task island mode is a transparent execution process that supports task interaction. When the user selects the task island mode, a task island card will be displayed on the automated task execution interface. The task island card displays at least one of the currently executing step information and functional interface information, and the process interface is the application interface of the currently executing task step.

[0067] See also Figure 3 , Figure 3 A process step display interface for creating a task island using the task island mode is provided in an embodiment of the present application. In this embodiment, when the user selects the task island mode, the display interface displayed by the electronic device when the first step is executed is the gallery interface. The electronic device starts the gallery application through the command signal and enters the gallery homepage for the user to select a portrait photo; the display page when the second step is executed is the gallery editing page. When starting the automated task, after selecting the portrait photo, the editing page of the portrait photo is entered. Figure 3(a) shows the display image of entering the gallery application homepage. In this image, in addition to the application interface of the gallery homepage itself, there is also a task island card displayed on the top side of the interface. The task island card contains the task step description of "select portrait photos", as well as the path information "Gallery> All Photos" from the main screen page to the current page, and a progress bar showing the completion of the task. The digital assistant icon is also shown in the image. Users can click on the corresponding control to select the corresponding portrait photo according to actual needs. Figure 3 (b) shows the portrait cutout interface. In addition to the existing controls for portrait operations (such as "Scale, Crop, Filter, Brightness, Recommend, Create, Send, Favorite, Delete, and More"), this interface also displays step descriptions, a progress bar indicating task completion, the corresponding path ("Gallery > Create"), and a task island icon. The execution process interface in task island mode not only helps users understand the progress of the task but also allows them to interact with the digital assistant in real time, making changes or guiding the digital assistant to better complete the task.

[0068] In a possible implementation of the present application, the execution progress bar of the task can be displayed with a circular progress bar, or it can be displayed with other shapes, as long as the user can perceive and clearly understand it. The display of the task island icon can also be varied. It can be displayed at the bottom of the interface, or it can be any position that supports user movement, or it can be hidden. The present invention aims to illustrate the interface display operation and the operation controls, operation paths, etc. displayed on the interface during the execution of automated tasks in the task island mode to help users better view and understand the specific implementation process of the task. The specific internal implementation algorithm and internal implementation logic are not described and elaborated here. The specific implementation of the operating system during actual use shall prevail.

[0069] In this embodiment, after the user completes the current step, he will proceed to the next step, and the task island card will also be updated with information matching the process interface of the next step. When executing a task step, the digital assistant may not complete all the task steps in one go, but may complete them step by step. After each task step is completed and before the next task step begins, the automated task will be in an interrupted state for a period of time. This period of time is used to display the information of the task island. In other words, when the task island card is displayed, the digital assistant does not perform any action in the background and is in a task interruption state. In addition, after the task step in the application interface is completed, the user can interact with the data assistant to enter the next task step. The way to enter the next task step can be through the voice command "next step" or using the operation control on the interface. If no input command is received from the interface, the digital assistant will automatically enter the next step after a preset time.

[0070] After a task step is completed and before the next step begins, the automated task is interrupted. In addition to displaying the task island card on the interface, a variety of task step execution methods are provided for users to choose from. One is that the user can manually execute the current action, and the other is that the user chooses to use a digital assistant to execute the current action. Therefore, the task island mode allows users to choose to execute according to their own wishes, or they can completely let the digital assistant execute automatically. This can be refined to the execution steps. During the execution of some task steps, users can choose to execute manually. In some unimportant task execution steps, users can completely let the digital assistant execute. This not only saves time but also better obtains the corresponding task execution results according to the user's wishes.

[0071] In one possible implementation scenario, in order to better complete the task, the digital assistant will ask for some additional information before executing the task steps to help the user better complete the corresponding task. The user can also actively instruct the digital assistant what kind of result he wants to get during the execution of this step by typing, numerically selecting controls, or inputting voice. The digital assistant will improve and adjust the corresponding task execution steps based on the user's intention information.

[0072] See also Figure 4 , Figure 4 A schematic diagram of an interface display of a task execution step supporting user interaction provided in an embodiment of the present application. In this embodiment, the interface displays the execution steps of a portrait cutout. Before this step is executed, the digital assistant asks, "I can help you make various ID photos. What specifications of ID photos do you need? The default is to make them according to the specifications of Chinese ID photos. Do you need help making them?", and then a "Please select the production method" dialog box will pop up on the interface with two operation controls, "Manual" and "AI production" for users to choose. Users can express their willingness to make them manually through voice input, such as "I will do it myself" as shown in the figure, or select the production method by touching the control on the interface, and when the electronic device does not receive any indication signal within the preset time (5s), it will directly and automatically execute the task step.

[0073] In one possible implementation of this embodiment, the task island card can be hidden and displayed during the execution of the task steps. The hiding and displaying methods include: not displaying in the interface, hiding against the edge, and displaying in the top status bar after performing at least one of shrinking and deforming. The methods of not displaying in the interface, hiding against the edge, and displaying in the top status bar after performing at least one of shrinking and deforming already exist in the prior art. This embodiment aims to help users better display the operation object in full screen or facilitate users to view the entire step operation process by hiding and displaying the task island card.

[0074] In one possible implementation, the user can also give voice instructions before or during the execution of the task step, such as "I want to make an ID card photo, please make it according to the specifications of the ID card photo." The digital assistant will perform semantic understanding and intent analysis based on the knowledge input by the user, and finally obtain the corresponding specification information of the ID card photo, and make it step by step according to the corresponding rules.

[0075] The above embodiments are flow charts of how a digital assistant divides automation tasks and executes them in different scenarios. However, this embodiment covers the implementation of all automation tasks and will not be listed here one by one.

[0076] See also Figure 5 , Figure 5 This is a schematic diagram of a digital opponent dialog box displayed in a task island mode provided by an embodiment of the present application. In this embodiment, it is mainly a schematic diagram of an automated task in the task island mode, wherein Figure 5 The schematic diagram on the left shows the task island card, operation object, operation control, digital assistant icon and the corresponding digital assistant's active inquiry content. The digital assistant asks, "I can help you make various ID photos. What specifications of ID photos do you need? The default is to make them according to the specifications of Chinese ID card photos. Do you need me to help you make them?" At this time, if the user wants to view the corresponding dialog box, he can directly swipe up the corresponding digital assistant, and then a dialog box will pop up on the interface. The dialog box will display all the conversation content between the digital assistant and the user. This function makes it easy for users to view previous conversation records, so that users can choose how to further execute the next step. In this embodiment, the user can swipe up the digital assistant at any time to view the corresponding dialog box, and help the user further in the subsequent execution steps according to the chat content in the dialog box, or use it to understand the subsequent detailed execution steps.

[0077] See also Figure 6 , Figure 6An electronic device provided in an embodiment of the present application includes a display screen and is equipped with a digital assistant, which is used for intelligent task guidance based on the digital assistant. The electronic device 100 includes a receiving module 101, a processing module 103, a display screen 105, etc., and is equipped with a digital assistant 107, wherein the receiving module 101 is used to receive user instructions and generate automated tasks, and the processing module 103 executes the automated task and obtains the execution result based on the selected automated task execution mode and automated step execution mode. The automated task execution mode includes a one-key generation mode and a task island mode, wherein the one-key generation mode is a mode in which the digital assistant 107 automatically executes the automated task in a black box; in the task island mode, the application interface corresponding to the current step will be displayed on the display screen 105, and a task island card will be displayed, which displays at least one of the execution path of the current step, the task island icon, the execution progress of the automated task, the operation control, and the operation object. The task island mode executes multiple steps in the automated task in a single step, and after each step is completed, the application interface corresponding to the current step remains displayed for a preset time. During this time, the display page of the application interface does not jump, and the current task step will be executed based on the user's intention only after the user enters an instruction. If no instruction from the user is received during this period, in the current step, the digital assistant executes the current task step in an automatic manner. Before each task step is executed, the digital assistant will stay for a while and actively ask the user's intention. The user can also actively make requests and suggestions to the digital assistant. The digital assistant will parse the user's intention according to the corresponding instructions and execute the current task step based on the user's intention.

[0078] In a possible implementation of this embodiment, for some display interfaces that do not want to be blocked or need to be displayed in full screen, during the execution of the automated task steps, the task island card can be presented in a hidden display manner, and the hidden display manner includes but is not limited to hiding it against the edge in the interface, not displaying it in the interface, and displaying it in the top status bar after performing at least one of shrinking and deforming it. This can generally be achieved by entering the next execution step, and the task island card will be displayed again or according to the user's instruction requirements. The display and hiding of the task island card, as well as the display method and hiding method, are not the subjects that need to be protected in this province. All display and hiding methods in the prior art are included in the protection scope of this application.

[0079] In one possible implementation of this embodiment, in the task island mode, the display screen shows the application interface corresponding to the current step, and a digital icon is displayed in the current application interface. The user can swipe up on the digital icon to display the digital assistant's dialog box, and the dialog box displays the conversation content between the digital assistant and the user. The user can view the conversation content in real time. This method can facilitate the user to view the execution steps of the entire automated task in real time based on the historical conversation content, and plan and consider the next execution step in advance.

[0080] In the task island mode, the digital assistant will interactively ask the user how to proceed next in some cases. At this time, a dialog box will be displayed on the display screen, and the corresponding controls will be displayed in the dialog box to facilitate the user to choose the next step. Therefore, in the task island mode, the control of how to execute the next step will often be displayed, and the control will at least include a control for instructing the user to execute manually. Therefore, the user can choose whether to execute manually in each task step, and can execute the corresponding task steps to a large extent according to the user's wishes, effectively improving the user experience.

[0081] Some of the functions or features described in the above embodiments may be implemented through software, some may be implemented through hardware, or some may be implemented through a combination of software, hardware, or firmware. When implemented through software, all or part of the functions or features may be implemented in the form of a computer program product.

[0082] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer language, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training equipment or data center to another website, computer, training equipment or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training equipment, data center, etc. that includes one or more available media integrations. Available media can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)).

Claims

1. An intelligent task guidance method based on a digital assistant, characterized in that: Applied to electronic devices with display screens and digital assistants, including: The electronic device receives user instructions and generates automated tasks; The automation task is executed based on the selected automation task execution mode and automation task step execution mode, and an execution result is obtained.

2. The method according to claim 1, characterized in that The automated task execution modes include a one-key generation mode and a task island mode.

3. The method according to claim 2, characterized in that The one-key generation mode is a mode in which the digital assistant black box executes the automation task; the task island mode is a mode in which the automation task is executed in a single step.

4. The method according to claim 2, characterized in that In the task island mode, the display screen displays the application interface corresponding to the current step and displays the task island card. The task island card displays at least one of the execution path of the current step of the automation task, the task island icon, the execution progress of the automation task, the operation control, and the operation object.

5. The method according to claim 4, characterized in that The task island mode executes multiple steps of the automation task in a single step, and after each step is executed, the application interface corresponding to the current step remains displayed within a preset time, and the task island card is displayed.

6. The method according to claim 4, characterized in that In the task island mode, during the execution of the steps of the automated task, the task island card is displayed in a hidden display manner, and the hidden display manner includes: not displaying in the interface, hiding along the edge, and displaying in the top status bar after performing at least one of shrinking and deforming.

7. The method according to any one of claims 4 to 6, characterized in that When the task island card is displayed, the display screen displays at least one control, and the at least one control is used to indicate the execution method of the next step of the automation task. The execution methods of the next step include manual and automatic, and the at least one control includes a manual execution control.

8. The method according to any one of claims 4 to 7, characterized in that: In the task island mode, the digital assistant icon is displayed in the application interface corresponding to the current step. Slide up the digital assistant icon to display the dialogue interface and dialogue content corresponding to the digital assistant.

9. The method according to claims 4-8, characterized in that In the process of executing the task steps of the automated task using the task island mode, the digital assistant executes the task steps based on the user's intention.

10. The method according to claim 9, characterized in that The user intention is obtained by the digital assistant based on at least one of active inquiry, receiving active input instructions from the user, and responding according to a predefined response after timeout.

11. The method according to claim 10, characterized in that The step of receiving the user's active input instruction is before executing the task step.

12. An electronic device comprising a display screen and equipped with a digital assistant, characterized in that: The electronic device is used for performing intelligent task guidance based on a digital assistant, including: A receiving module, used to receive user instructions and generate automated tasks; The processing module executes the automation task based on the selected automation task execution mode and the automation task step execution mode and obtains the execution result.

13. The electronic device according to claim 12, wherein: The automated task execution modes include a one-key generation mode and a task island mode.

14. The electronic device according to claim 13, wherein: The one-key generation mode is a mode in which the digital assistant black box executes the automation task; the task island mode is a mode in which the automation task is executed in a single step.

15. The electronic device according to claim 13, wherein: In the task island mode, the display screen displays the application interface corresponding to the current step and displays the task island card. The task island card displays at least one of the execution path of the current step of the automation task, the task island icon, the execution progress of the automation task, the operation control, and the operation object.

16. The electronic device according to claim 14, characterized in that The task island mode executes multiple steps of the automation task in a single step, and after each step is executed, the application interface corresponding to the current step remains displayed within a preset time, and the task island card is displayed.

17. The electronic device according to claim 14, wherein: In the task island mode, during the execution of the steps of the automated task, the task island card is displayed in a hidden display manner, and the hidden display manner includes: not displaying in the interface, hiding along the edge, and displaying in the top status bar after performing at least one of shrinking and deforming.

18. The electronic device according to any one of claims 14 to 17, characterized in that: When the task island card is displayed, the display screen displays at least one control, and the at least one control is used to indicate the execution method of the next step of the automation task. The execution methods of the next step include manual and automatic, and the at least one control includes a manual execution control.

19. The electronic device according to any one of claims 14 to 18, characterized in that: In the task island mode, the digital assistant icon is displayed in the application interface corresponding to the current step. Slide up the digital assistant icon to display the dialogue interface and dialogue content corresponding to the digital assistant.

20. The electronic device according to claims 14-19, characterized in that: In the process of executing the task steps of the automated task using the task island mode, the digital assistant executes the task steps based on the user's intention.

21. The electronic device according to claim 20, characterized in that The user intention is obtained by the digital assistant based on at least one of active inquiry, receiving active input instructions from the user, and responding according to a predefined response after timeout.

22. The electronic device according to claim 21, wherein: The step of receiving the user's active input instruction is before executing the task step.

23. An electronic device with a display interface, characterized in that: The electronic device comprises a processor coupled to a memory, wherein the processor is configured to store instructions. When the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on a computer system, enable the computer system to implement the method according to any one of claims 1 to 11.

25. A computer program product comprising instructions, characterized in that: When the instructions are executed, the computer implements the method according to any one of claims 1 to 11.

Images