Information display method and device

Abstract

The invention discloses an information display method and device, and belongs to the technical field of terminals. Receiving a task description prompt word input by the user, wherein the task description prompt word comprises task description information of the first task; executing the first task based on the task description cue word; and displaying at least one of task progress description information for describing the current task progress of the first task, an application identifier of an application program currently executing the first task, and related information of a currently completed task node of the first task in the first area.

Description

Technical Field

[0001] This application belongs to the technical field of terminals, and particularly relates to an information display method and its device. Background Art

[0002] With the rapid development of electronic devices and the Internet, people increasingly rely on electronic devices to complete a large number of tasks in life. Currently, major electronic device manufacturers have gradually launched artificial intelligence (AI) assistants to implement various automatic operation functions, so that users do not need to perform a large number of repetitive operations on electronic devices, saving time and effort.

[0003] Generally, after an electronic device triggers an automatic operation function, it can perform automatic operations through the AI assistant. At this time, the electronic device can display the real-time progress of the automatic operation in the foreground. However, when the electronic device displays the real-time progress of the automatic operation in the foreground, it usually shows a full screen. At this time, if the user wants to manually operate the electronic device, the electronic device defaults to turning off the automatic operation function, resulting in the inability to complete the task of the automatic operation and further leading to task failure. In this way, the ability of the electronic device to execute tasks in parallel is poor. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide an information display method and its device, which can improve the ability of an electronic device to execute tasks in parallel.

[0005] In a first aspect, the embodiments of this application provide an information display method, which includes: receiving a task description prompt word input by a user, where the task description prompt word includes task description information of a first task; executing the first task based on the task description prompt word; and displaying at least one of the following in a first area: task progress description information for describing the current task progress of the first task, an application identifier of an application currently executing the first task, and relevant information of a task node that the first task has currently completed.

[0006] In a second aspect, the embodiments of this application provide an information display device, which includes: a receiving module, a processing module, and a display module. The receiving module is used to receive a task description prompt word input by a user, where the task description prompt word includes task description information of a first task. The processing module is used to execute the first task based on the task description prompt word received by the receiving module. The display module is used to display at least one of the following in a first area: task progress description information for describing the current task progress of the first task, an application identifier of an application currently executing the first task, and relevant information of a task node that the first task has currently completed.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores a program or instructions that can run on the processor. When the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored. When the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the method described in the first aspect.

[0011] In the embodiment of the present application, the electronic device can receive a task description prompt word input by a user, and the task description prompt word includes task description information of a first task; and based on the task description prompt word, execute the first task; and display at least one of the following in a first area: task progress description information for describing the current task progress of the first task, an application identifier of the application program currently executing the first task, and relevant information of the task nodes that have been completed for the first task currently. In this solution, after receiving the task description prompt word input by the user, the electronic device can analyze the task description prompt word to execute the task indicated by the task description prompt word, and display the task-related information of the task in the first area, so that the execution progress of the task can be viewed in real time without affecting the user's operation of the electronic device, improving the ability of the electronic device to execute tasks in parallel and the human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is one of the flow schematic diagrams of the information display method provided by the embodiment of the present application;

[0013] Figure 2 is an example schematic diagram of the electronic device receiving a task description prompt word provided by the embodiment of the present application;

[0014] Figure 3 is an example schematic diagram of the first area in the default display state provided by the embodiment of the present application;

[0015] Figure 4 is an example schematic diagram of the first area in the expanded display state provided by the embodiment of the present application;

[0016] Figure 5 It is an example schematic diagram of a page screenshot of an operation page of a task node provided by an embodiment of the present application;

[0017] Figure 6 It is an example schematic diagram of displaying a first screenshot in a first area provided by an embodiment of the present application;

[0018] Figure 7A It is one of the example schematic diagrams of a front-end of an electronic device executing a first task provided by an embodiment of the present application;

[0019] Figure 7B It is another example schematic diagram of a front-end of an electronic device executing a first task provided by an embodiment of the present application;

[0020] Figure 7C It is one of the example schematic diagrams of an electronic device displaying a second screenshot provided by an embodiment of the present application;

[0021] Figure 7D It is another example schematic diagram of an electronic device displaying a second screenshot provided by an embodiment of the present application;

[0022] Figure 8 It is the second flowchart schematic diagram of an information display method provided by an embodiment of the present application;

[0023] Figure 9 It is a schematic structural diagram of an information display device provided by an embodiment of the present application;

[0024] Figure 10 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0025] Figure 11 It is a schematic hardware diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0027] The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the description and claims means at least one of the connected objects. The character " / ", generally represents an "or" relationship between the associated objects before and after.

[0028] The terms "at least one (item)", "at least one of", etc. in the description and claims of this application refer to any one, any two or more combinations of the objects it contains. For example, at least one (item) of a, b, and c can represent: "a", "b", "c", "a and b", "a and c", "b and c", and "a, b, and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" means two or more, and its meaning is similar to that of "at least one (item)".

[0029] The following will combine the accompanying drawings and, through specific embodiments and their application scenarios, elaborate in detail on the information display method and device provided by the embodiments of this application.

[0030] The technical solution provided by the embodiments of this application can be applied to scenarios where an electronic device performs automatic operations through an AI assistant. The following takes some specific scenarios of the embodiments of this application as examples to exemplarily illustrate the information display method provided by the embodiments of this application.

[0031] Scenario 1, the user needs to view the task progress information of the task of the electronic device purchasing coffee through the AI assistant

[0032] The user can enter the information "Help me order a large cup of AA's raw coconut latte, iced, without extra sugar" in the AI interaction interface. This information instructs the AI assistant to purchase the coffee needed by the user online. Then, the electronic device can execute the coffee purchase task through the AI assistant based on the information input by the user. Next, the electronic device can display "Task in progress" in the notification bar area.

[0033] Scenario 2, the user needs to view the task progress information of the task of the electronic device purchasing medicine through the AI assistant

[0034] The user can input the information "Help me buy a box of medicine CC for treating colds, with a specification of 1 box containing 9 bags" in the AI interaction interface. This information instructs the AI assistant to purchase the medicine needed by the user online. Then, the electronic device can execute the medicine purchase task through the AI assistant based on the information input by the user. Next, the electronic device can display "Task in progress" and the application icon of the application for the medicine purchase task in the notification bar area.

[0035] Scenario 3, the user needs to view the task progress information of the task of the electronic device purchasing train tickets through the AI assistant.

[0036] The user can input the information "Help me buy a train ticket from City A to City B on April 5th, with the earliest departure time and an upper berth" in the AI interaction interface. This information instructs the AI assistant to purchase the train ticket needed by the user online. Then, the electronic device can execute the train ticket purchase task through the AI assistant based on the information input by the user. Next, the electronic device can display "Task in progress" and the application icon of the application for the train ticket purchase task in the notification bar area.

[0037] It should be noted that the above Scenarios 1 to 3 only exemplarily list some scenarios where the embodiments of the present application may be applied. In actual implementation, the embodiments of the present application can also be applied to more scenarios, and the embodiments of the present application are not limited thereto.

[0038] For the information display method and its device provided by the embodiments of the present application, after receiving the task description prompt word input by the user, the electronic device can analyze the task description prompt word to execute the task indicated by the task description prompt word, and display the task-related information of the task in the first area, so that the execution progress of the task can be viewed in real time without affecting the user's operation of the electronic device, improving the ability of the electronic device to execute tasks in parallel and the human-computer interaction.

[0039] For the information display method provided by the embodiments of the present application, the execution subject can be an information display device. This information display device can be an electronic device, or a functional module and an entity module in the electronic device that can implement the information display method, which can be specifically determined according to actual usage requirements, and the embodiments of the present application are not limited thereto. Hereinafter, taking the electronic device executing this information display method as an example, the information display method provided by the embodiments of the present application will be described.

[0040] The embodiments of the present application provide an information display method. Figure 1 The flowchart showing the information display method provided by the embodiments of the present application is as follows. As Figure 1 shown, the information display method provided by the embodiments of the present application may include steps 201 to 203.

[0041] Step 201, the electronic device receives a task description prompt word input by the user.

[0042] Among them, the task description prompt word input by the user may include the task description information of the first task.

[0043] In some embodiments of the present application, the user may input the task description prompt word into the electronic device by manually inputting the task description prompt word, and the user may also input the task description prompt word into the electronic device by voice inputting the task description prompt word, which can be specifically determined according to actual usage requirements.

[0044] In some embodiments of the present application, the task description prompt word may also be referred to as requirement clarification.

[0045] In some embodiments of the present application, the task description prompt word may be in the form of natural language.

[0046] For example, in the above scenario 1, the task description prompt word may be: "Help me order a large cup of AA's coconut latte, iced, without extra sugar".

[0047] For another example, in the above scenario 2, the task description prompt word may be: "Help me buy a box of medicine CC for treating colds, with a specification of 1 box of 9 bags".

[0048] For still another example, in the above scenario 3, the task description prompt word may be: "Help me buy a train ticket from City A to City B on April 5, with the earliest departure time, upper berth".

[0049] In some embodiments of the present application, the user may first trigger the electronic device to start the AI assistant by input, and input the task description prompt word in the AI interaction interface.

[0050] In some embodiments of the present application, the task in the present application may be understood as a task that the electronic device needs to execute to complete a specific goal.

[0051] For example, the task may include any one of the following: the task of buying coffee, the task of buying medicine, the task of buying a train ticket, the task of opening an application, the task of backing up files, the task of setting a schedule reminder, the task of sending a message, the task of pushing a message, the task of cleaning up the memory, the task of automatically playing games, etc.

[0052] It should be noted that the above task of buying coffee may also be referred to as the coffee purchase task, the task of buying medicine may also be referred to as the medicine purchase task, the task of buying a train ticket may also be referred to as the train ticket purchase task, etc.

[0053] In some embodiments of the present application, the task description information of the above task is used to describe the task that the user wants the AI assistant to execute.

[0054] Exemplarily, the task description information of the above task may include at least one of the following: the program name of the application that the electronic device needs to execute for this task, the task key content information of the task, and the name of the task.

[0055] Step 202: The electronic device executes a first task based on the task description prompt entered by the user.

[0056] In some embodiments of the present application, after receiving the task description prompt entered by the user, the electronic device may obtain the task description information of the first task from the task description prompt, and then execute the first task based on the task description information of the first task.

[0057] In some embodiments of the present application, the electronic device can analyze the above task description prompt through an AI assistant to obtain the task description information of the first task included in the analysis of the task description prompt, and then execute the first task based on the task description information of the first task.

[0058] In some embodiments of the present application, after completing the analysis of the above task description prompt, the electronic device may display a task confirmation window in the AI interaction interface, and the task confirmation window may include at least one of the following: the task key content information corresponding to the task description information of the first task, the task confirmation information of the first task, a confirmation control, and a modification control.

[0059] In some embodiments of the present application, the user can input to the above confirmation control to trigger the electronic device to start executing the first task.

[0060] In some embodiments of the present application, the user can input to the above modification control to operate the electronic device to modify the key content information corresponding to the task description information of the first task in the above task confirmation window, and after the user operates the electronic device to modify the key content information, the electronic device can start executing the first task.

[0061] The following will give an exemplary illustration of the above step 202 in conjunction with specific drawings.

[0062] Example 1, in combination with the above Scenario 1, as Figure 2As shown, the user can trigger the electronic device to start the AI assistant by voice inputting "Open AI assistant". Then, the user can voice input the following information 21 (i.e., the task description prompt) in the AI interaction interface: "Help me order a large cup of AA's coconut latte, iced, without extra sugar". Next, the electronic device can analyze this information 21 through the AI assistant to obtain the task description information of the coffee purchase task included in the analysis of this information 21, and display a task confirmation window 22 in the AI interaction interface. It can be seen that the task confirmation window 22 includes: the key content information 24 corresponding to the task description information of the coffee purchase task, the task confirmation information 23 of the coffee purchase task, a confirmation control 25, and a modification control 26. After the user clicks the confirmation control 25, the electronic device can execute the coffee purchase task based on the task description information of the coffee purchase task. If the user clicks the modification control 26, the user can voice input, such as "Change the large cup to a medium cup", to trigger the electronic device to modify the task description information of the coffee purchase task. After the user finishes inputting, the electronic device can start to execute the coffee purchase task.

[0063] Step 203: The electronic device displays at least one of the following in the first area: task progress description information for describing the current task progress of the first task, the application identifier of the application currently executing the first task, and relevant information of the task nodes that the first task has currently completed.

[0064] In some embodiments of the present application, the first area may be a partial display area in the display area of the electronic device screen.

[0065] Exemplarily, the first area may be an area of the status bar.

[0066] In some embodiments of the present application, the first area may be a movable area. That is, after the user inputs to the first area, it can trigger the electronic device to move the first area.

[0067] In some embodiments of the present application, the first area may include an interaction component.

[0068] In some embodiments of the present application, the above-mentioned interaction component may be a control that does not depend on a certain application in the electronic device. That is, the above-mentioned interaction component can be displayed in any interface of the electronic device.

[0069] In some embodiments of the present application, the above-mentioned interaction component may be located at the top layer of the screen layer.

[0070] Exemplarily, the above-mentioned interaction component may also be referred to as an atomic island.

[0071] In some embodiments of the present application, the shape of the above-mentioned interaction component can be any possible shape such as a circle, rectangle, triangle, rhombus, ring, or polygon. Specifically, it can be determined according to actual usage requirements, and the embodiments of the present application do not make any limitations.

[0072] In some embodiments of the present application, the above-mentioned interaction component can be a floating control.

[0073] Exemplarily, the above-mentioned interaction component can be floatingly displayed at any position on the screen. When the user drags the interaction component, the interaction component can move on the screen along with the user's dragging operation.

[0074] In some embodiments of the present application, the above-mentioned interaction component can be fixedly displayed at a certain position on the screen.

[0075] Exemplarily, the above-mentioned interaction component can be fixedly displayed at a preset position on the screen of the electronic device.

[0076] For example, the above-mentioned preset position can be any position on the screen of the electronic device. For example, the above-mentioned preset position can include at least one of the following: status bar, notification bar, task bar, virtual navigation bar.

[0077] Preferably, the preset position on the above-mentioned screen can be the position around the front camera on the screen.

[0078] Exemplarily, as Figure 3 shown, a component 30, that is, the above-mentioned interaction component, can be displayed in the status bar of the screen.

[0079] In some embodiments of the present application, the task progress description information of the current task of the first task can include: the preset task progress description information of the preset task progress in the electronic device, or the description information of the task execution action of the current task of the electronic device.

[0080] In some embodiments of the present application, the above-mentioned preset task progress description information can be the content preset in the electronic device. It can be understood that multiple preset task progress description information are pre-stored in the electronic device, and different preset task progress description information are used to describe different states of the task progress.

[0081] Exemplarily, the above-mentioned preset task progress description information can be pictures, texts, icons, etc.

[0082] Exemplarily, the above-mentioned preset task progress description information can be text, and the text can be a fixed formula, for example, "Task in progress", "Task completed", etc.

[0083] Exemplarily, the above-mentioned preset task progress description information may be a picture, which may contain information capable of indicating the task progress, such as a dynamic picture containing a task progress bar.

[0084] Exemplarily, the above-mentioned preset task progress description information may be an icon. For example, the icon may be a task progress bar identifier, which can use colors to distinguish the completed part and the uncompleted part of the task, as well as the task progress during the task execution process.

[0085] For example, when the left half and the right half of the task progress bar identifier have different colors, it may indicate that the current task has been executed halfway and there is still half left unexecuted, and the task progress of the current task is 1 / 2; when the color of the left 1 / 3 of the task progress bar is different from the color of the right 2 / 3, it may indicate that the current task has been executed 1 / 3 and there is still 2 / 3 left unexecuted, and the task progress of the current task is 1 / 3.

[0086] In some embodiments of the present application, the description information of the task execution action of the current task may be understood as: the action description information of the execution action when the electronic device executes the current task.

[0087] For example, if the task currently executed by the electronic device is a coffee purchase task, the description information of the task execution action of the current task in the electronic device may be "enter the AA applet", "click on the search box", "enter 'coconut latte'", etc.

[0088] In some embodiments of the present application, the application identifier of the application program may include any one of the following: the application icon of the application program, the application name of the application program.

[0089] It can be understood that the application identifier of the application program currently executing the first task may include: the application identifier corresponding to the application program used by the electronic device when executing the first task currently.

[0090] For example, if the application program used by the electronic device to execute the coffee purchase task in the above scenario 1 is Application 1, then the application icon of Application 1 may be displayed in the first area.

[0091] In some embodiments of the present application, the relevant information of the task node that the first task has currently completed may include at least one of the following: the node identifier of the task node that the first task has currently completed, the action description information of the execution action corresponding to the task node that the first task has currently completed.

[0092] In some embodiments of the present application, a task node may be understood as an execution action during the process of the electronic device executing a task, and one task node corresponds to one execution action.

[0093] For example, when the electronic device executes the coffee purchase task in Scenario 1, "Open Mini Program AA" is a task node, and "Click on the search box" is the next task node of this task node.

[0094] In some embodiments of the present application, a task node may correspond to action description information of an execution action.

[0095] It can be understood that the action description information of the execution action can be used to concisely summarize the actions performed by the electronic device.

[0096] For example, in the above Scenario 1, the action description information of the execution actions of the coffee purchase task may be: "Click on the search box", "Click on the back button", "Input: Iced Coconut Latte", "Return to the previous menu", etc.

[0097] In some embodiments of the present application, the manifestation forms of the action description information of the execution action may include: text form, symbol form, icon form, etc., any possible manifestation form, and the embodiments of the present application do not limit this.

[0098] In some embodiments of the present application, the node identifier of a task node may have an association relationship with the action description of the execution action corresponding to the task node, and the node identifier of a task node is associated with the action description of the execution action corresponding to this task node.

[0099] In some embodiments of the present application, the node identifier of the task node that the first task has currently completed can be understood as: all the node identifiers of the task nodes that the first task has currently completed, or the node identifiers corresponding to at least the last task nodes of the task nodes that the first task has currently completed.

[0100] In some embodiments of the present application, the action description information of the execution actions corresponding to the task nodes that the first task has currently completed may correspond one-to-one with the node identifiers of the task nodes that the first task has completed in the first area. In other words, the number of node identifiers of the task nodes that the first task has completed and are displayed in the first area: is the same as the number of action description information of the execution actions corresponding to the task nodes that the first task has currently completed, and one node identifier corresponds to one action description information.

[0101] In the information display method provided by the embodiments of the present application, after receiving the task description prompt word input by the user, the electronic device can analyze the task description prompt word to execute the task indicated by the task description prompt word, and display the task-related information of the task in the first area, so that the execution progress of the task can be viewed in real time without affecting the user's operation of the electronic device, improving the ability of the electronic device to execute tasks in parallel and the human-computer interaction.

[0102] In some embodiments of the present application, step 202 can be specifically implemented through the following step 202A and step 202B.

[0103] Step 202A: The electronic device obtains the task description information of the first task from the task description prompt words input by the user.

[0104] In some embodiments of the present application, the electronic device can, through an AI assistant, identify the keywords or keyword information in the above task description prompt words, analyze the identified keywords or keyword information, so as to obtain the task description information of the first task.

[0105] For example, when the electronic device executes the coffee purchase task in scenario 1, assuming that the task description prompt word for the coffee purchase task is: "Help me order a large cup of AA's coconut latte, iced, without extra sugar". Then, the electronic device can identify the keywords in the task description prompt word for the coffee purchase task: "order a cup", "AA", "coconut latte", "large cup", "iced", "without extra", "sugar", analyze the identified keywords, and thus can obtain the task description information of the coffee purchase task. Among them, the task description information of the coffee purchase task may include: this task is a coffee purchase task, and the key content information of this task is: the number of coffees purchased is one cup, the coffee purchased is coconut latte, the cup size of the coffee purchased is large cup, the temperature requirement of the coffee purchased is iced, the sugar content requirement of the coffee purchased is without extra sugar, and the manufacturer of the coffee purchased is AA.

[0106] Step 202B: The electronic device executes the first task based on the task description information of the first task.

[0107] In some embodiments of the present application, the electronic device can generate the link information of the task node link of the first task based on the above task description information of the first task, and execute the first task based on the link information.

[0108] Among them, the task node link of the first task may include at least one task node of the first task, and the link information may include the action description information of the execution action corresponding to each task node.

[0109] In some embodiments of the present application, the task node link of the first task may include what the execution actions are when the electronic device executes the first task, and the sequence of execution actions during the execution of the first task.

[0110] In some embodiments of the present application, the above link information can be used to represent the execution actions that need to be executed during the process of the electronic device executing the first task.

[0111] In some embodiments of the present application, the electronic device may request the end - side large model with a prompt message, and the end - side large model generates the task node link of the above - mentioned first task and the link information of the task node link of the above - mentioned first task. In other words, the electronic device may request the end - side large model with a prompt, and the large model returns the automatic operation actions (i.e., the execution actions corresponding to each task node in the task node link) and the action description information (i.e., the above - mentioned link information).

[0112] Among them, the end - side large model can be understood as an artificial intelligence (AI) model built into the electronic device.

[0113] For example, the action definitions of the execution actions corresponding to at least one task node included in the above - mentioned task node link may be as shown in Table 1 below:

[0114] Table 1

[0115]

[0116]

[0117] In some embodiments of the present application, for the click action, input action, and slide action in Table 1, there are corresponding interaction area parameters in the action parameters, and one execution action corresponds to one interaction area.

[0118] Among them, "area: list" in Table 1 represents the area range, "text: str" represents the text string, and "app corresponding package name" represents the application name of the application (application, app).

[0119] Among them, the examples in Table 1 are the codes for exemplifying the action parameters.

[0120] It should be noted that the above - mentioned Table 1 is an exemplary definition of the execution actions of the task. In actual implementation, there may be more definitions for the execution actions of the task, and the embodiments of the present application do not limit this.

[0121] It can be understood that the electronic device may execute the first task according to the task node link of the first task based on the link information of the task node link of the first task.

[0122] The following uses specific examples to illustrate the above - mentioned step 202A and step 202B.

[0123] Example 2, combining the above - mentioned scenario 1 and Example 1, when the user clicks as Figure 2After the confirmation control 25 shown, the electronic device may request the end-side large model to generate link information of the task node link for the coffee purchase task: "Open App 1" → "Enter Mini Program AA" → "Click on the search box" → "Enter 'Coconut Latte'" → "Select parameter information: large cup, ice, no extra sugar" → "Click to purchase" → "Click to confirm payment" → "Display the payment interface". Then, the electronic device may execute the coffee purchase task based on this link information according to the execution actions included in the task node link: "Open App 1 → Enter Mini Program AA → Click on the search box → Enter 'Coconut Latte' → Select parameter information: large cup, ice, no extra sugar → Click to purchase → Click to confirm payment → Display the payment interface".

[0124] In this way, the electronic device can execute the first task based on the obtained task description information of the first task without the user manually operating the electronic device to execute the first task, thereby improving the accuracy and operational convenience of the electronic device in executing tasks.

[0125] It should be noted that in the embodiments of the present application, the link information of the task node link of the first task is generated by the electronic device for illustration. In actual implementation, the electronic device may generate one task node of the first task at a time and execute the execution action corresponding to one task node until the first task is completed; or, the electronic device may complete the first task in other ways of generating task nodes and executing the first task. For example, the electronic device may generate at least two task nodes of the first task at a time and sequentially execute the execution actions corresponding to the at least two task nodes until the first task is completed. The embodiments of the present application do not limit this.

[0126] In some embodiments of the present application, step 203 may be specifically implemented through the following step 203A.

[0127] Step 203A: During the execution of the first task by the electronic device, task progress description information for describing the current task progress of the first task and the application identifier of the application program currently executing the first task are displayed in the first area.

[0128] Among them, the task progress description information of the current task progress of the first task can be used to describe the current task progress of the first task.

[0129] In some embodiments of the present application, the first area may have two display states, and the two display states are respectively the default display state and the expanded display state.

[0130] In some embodiments of the present application, in the default display state, during the execution of the first task by the electronic device, task progress description information for describing the current task progress of the first task and the application identifier of the application program currently executing the first task may be displayed in the first area.

[0131] In some embodiments of the present application, the above default display state may be the initial display form of the task progress information of the first task displayed in the above first area, for example Figure 3 the first area 30 shown.

[0132] In some embodiments of the present application, for the description of the first area in the expanded display state, reference may be made to the relevant description in the following embodiments.

[0133] It should be noted that the task progress description information of the current task progress of the above first task may refer to the detailed description in the above text. To avoid repetition, it will not be elaborated here.

[0134] Next, with reference to the accompanying drawings, the above step 203A will be exemplarily described.

[0135] Example 3, in combination with the above Scenario 1, when the electronic device is executing the coffee purchase task and the first area is in the default display state, the electronic device may display in the first area 30 as shown Figure 3 the fixed words "Task in progress" 31 (i.e., the preset task progress description information), and the application icon 32 of the application 1 currently executing the coffee purchase task. It can be seen that at this time, the first area 30 is displayed in a reduced form on the current screen of the electronic device, and the electronic device displays the application icon 32 of the application program for the current coffee purchase task and the fixed words "Task in progress" 31 in the first area 30.

[0136] In this way, by displaying the task progress description information of the current task progress of the first task and the application identifier of the application program currently executing the first task in the first area, it is possible to enable the user to know the execution status of the first task and which application program is executing the first task through this first area without affecting the user's operation of the electronic device, improving the ability of the electronic device to execute tasks in parallel and the human-computer interaction.

[0137] In some embodiments of the present application, the above step 203 may be specifically implemented through the following step 203B.

[0138] Step 203B: During the execution of the first task by the electronic device, in the first area, display at least one of the following:

[0139] The task progress description information used to describe the current task progress of the first task;

[0140] The application identifier of the application program currently executing the first task;

[0141] The relevant information of the task nodes that have been completed in the first task currently.

[0142] In some embodiments of the present application, during the execution of the first task, the electronic device may display any one, any two, or all three of the following three items in the first area:

[0143] 1) Task progress description information for describing the current task progress of the first task;

[0144] 2) The application identifier of the application currently executing the first task;

[0145] 3) Information related to the task nodes that have been completed in the first task currently.

[0146] In some embodiments of the present application, the user can trigger the electronic device to update the first area from the default display state to the expanded display state by inputting into the above-mentioned first area.

[0147] In some embodiments of the present application, when the first area is in the expanded display state, at least one of the following may be displayed in the first area: task progress description information for describing the current task progress of the first task; the application identifier of the application currently executing the first task; information related to the task nodes that have been completed in the first task currently.

[0148] In some embodiments of the present application, the area size of the first area is different when the first area is in the default display state and the expanded display state.

[0149] Exemplarily, when the first area is in the default display state, the area size of the first area is smaller than: the area size of the first area when the first area is in the expanded display state. In other words, when the first area is in the default display state, the display area size of the first area is smaller than: the display area size of the first area when the first area is in the expanded display state.

[0150] The following will exemplarily illustrate the above step 203B with reference to the accompanying drawings.

[0151] Exemplarily, in combination with the above scenario 1, assume that the content displayed in the first area includes the above items 1) to 3), the first task is a coffee purchase task, and the application for executing the coffee purchase task is application 1. During the execution of the coffee purchase task by the electronic device and when the first area is in the expanded display state, the user can click on the first area 30 as shown in Figure 3 to trigger the electronic device to update the display state of the first area 30 from the default display state to: the expanded display state of the first area 40 as shown in Figure 4 shown.

[0152] It can be seen that when the first area 40 in Figure 4 is in the expanded display state, the display area size of the first area 40 is larger than that in Figure 3The regional size of the display area in the first region 30 shown, and in the first region 40, the following can be displayed: the fixed dialogue "Task in progress" 41 (i.e., the above-mentioned preset task progress description information), the application identifier 42 of application 1, the node identifier 43 of the task node that has been completed in the coffee purchase task currently, and the action description information 44 of the execution action corresponding to the task node that has been completed in the coffee purchase task currently.

[0153] Among them, in Figure 4 the left knot-shaped identifier is the node identifier of the task node. To the right of each black dot is the action description information of the execution action corresponding to the task node identifier. The task nodes that have been completed in the coffee purchase task currently are: "Enter the AA mini-program", "Click on the search box", and "Enter 'Virgin Coconut Latte'". The electronic device displays the steps (i.e., execution actions) that have been completed in the current coffee purchase task in the form of node identifiers in the first region.

[0154] In some embodiments of the present application, the user can, through the node information of the first task displayed in the first region (i.e., the node identifier of the task node that has been completed in the first task currently and the action description information of the execution action corresponding to the task node that has been completed in the first task currently), master and monitor the real-time progress of the electronic device executing the first task, clearly understand the execution actions that have been completed in the first task, and know whether there is a risk of leakage of personal privacy information. Thereby, the information display effect of the electronic device can be improved, and the security of the user's private data can be protected. Also, displaying the relevant information of the first task in the first region does not interfere with the user's normal operation of the electronic device. Thus, the user can both experience the convenience of the automatic operation of the electronic device (i.e., automatically execute tasks) and freely operate the electronic device, saving waiting time. Therefore, the ability of the electronic device to execute tasks in parallel and the operation flexibility can be improved.

[0155] In this way, by displaying the task progress description information of the task progress of the first task, the application identifier of the application program currently executing the first task, and the relevant information of the task node that has been completed in the first task currently in the first region, it is possible to enable the user to know the detailed progress information of the execution progress of the first task and which application program is executing the first task through this first region without affecting the user's operation of the electronic device, improving the ability of the electronic device to execute tasks in parallel and the human-computer interaction.

[0156] In some embodiments of the present application, after the above step 201, the information display method provided by the embodiments of the present application may include the following step 301, and specifically, the above step 202 may be implemented through the following step 202C.

[0157] Step 301: The electronic device starts a virtual screen in the background.

[0158] It can be understood that the virtual screen is started in the background of the electronic device, and the user can perform any operation on the front screen of the electronic device.

[0159] In some embodiments of the present application, the electronic device can project the front page onto the virtual screen, so that the page information on the virtual screen is consistent with the page information on the front page.

[0160] Step 202C: The electronic device executes a first task in the virtual screen based on the task description prompt word input by the user.

[0161] In some embodiments of the present application, after the user inputs the above task description prompt word, the electronic device can analyze the task description prompt word, start the virtual screen, and execute the above first task in the virtual screen.

[0162] In some embodiments of the present application, the electronic device can generate link information of the task node link of the above first task based on the above task description prompt word input by the user, and execute the above first task in the virtual screen based on the link information of the task node link of the above first task.

[0163] Exemplarily, in combination with the above scenario 1 and example 1, after the user clicks the confirmation control 25 as shown in Figure 2 , the electronic device can start the virtual screen in the background and project the current page of the electronic device onto the virtual screen. Then, the electronic device can take a screenshot of the current page of the virtual screen to obtain a screenshot image of the current page, and obtain the Extensible Markup Language (XML) information in the virtual screen through the accessibility interface in the electronic device. Next, the electronic device can combine the screenshot image and the XML information into a prompt, request the side large model with the prompt, and after the side large model generates the link information of the task node link of the coffee purchase task, execute the coffee purchase task based on the link information.

[0164] Among them, the result of the prompt can be as follows:

[0165] You are an automatic operation assistant (i.e., an AI assistant) of an electronic device. You need to judge the specific action of the next automatic operation (i.e., the execution action corresponding to the next task node) according to the task objective and the current page information.

[0166] Current task objective: aaaaa

[0167] Current page XML information: bbbbb

[0168] Current page picture: ccccc

[0169] Please give the next action and action description...

[0170] In some embodiments of the present application, the electronic device may execute the above-mentioned first task on the virtual screen based on the above-mentioned link information generated by the end-side large model.

[0171] Specifically, the electronic device may execute the corresponding action (i.e., the execution action corresponding to the task node of the above-mentioned first task) on the above-mentioned virtual screen based on the action type (i.e., the execution action corresponding to the above-mentioned task node) and the corresponding action parameters generated by the end-side large model.

[0172] For example, if the action type output by the end-side large model is "input" in Table 1 above and the action parameter is {'area': [100, 100, 150, 100], 'text': 'Latte'}, it means that the electronic device needs to input "Latte" in the area with coordinates [100, 100, 150, 100] on the virtual screen.

[0173] In this way, the electronic device can start the virtual screen and execute the first task on the virtual screen based on the task description prompt entered by the user. Since the first task executed on the virtual screen will not be displayed on the foreground interface of the electronic device, the user can also manually operate the electronic device to enable other functions. Therefore, the ability of the electronic device to execute tasks in parallel and the operation flexibility can be improved.

[0174] In some embodiments of the present application, after the above-mentioned step 301, the information display method provided by the embodiments of the present application may include the following steps 302 and 303.

[0175] Step 302: During the process of the electronic device executing the first task on the virtual screen, take a screenshot of the operation page of each task node in the first task to obtain a screenshot of the operation page of each task node.

[0176] Step 303: The electronic device stores the screenshots of the operation pages of each task node.

[0177] In some embodiments of the present application, during the process of the electronic device executing the above-mentioned first task on the above-mentioned virtual screen, after completing the execution action corresponding to a task node of the above-mentioned first task on the above-mentioned virtual screen, the electronic device may take a screenshot of the operation page of the task node on the above-mentioned virtual screen to obtain a screenshot of the operation page of the task node, and store the screenshot of the operation page of the task node.

[0178] For example, assume that the first task is the coffee purchase task in Scenario 1, and the task node of the first task executed by the electronic device in the virtual screen is to select coffee parameter information. Then, after the electronic device completes the selection of coffee parameter information for the coffee purchase task in the virtual screen, it can take a screenshot of the operation page of the task node for selecting coffee parameter information, and obtain a page screenshot 51 of the operation page of this task node as shown in Figure 5 and store the page screenshot 51 in the album application.

[0179] In some embodiments of the present application, the electronic device can associate the image identifier of the page screenshot of the operation page of the task node with the task node, so that the electronic device can obtain the page screenshot of the operation page of the task node based on the association relationship between the image identifier and the task node.

[0180] In some embodiments of the present application, the electronic device can also display in the above-mentioned first area: the page screenshot of the operation page of the task node corresponding to the completed task node.

[0181] In this way, since the electronic device can map the execution process of the first task to the virtual screen, the electronic device can take a screenshot of the virtual screen to obtain and store the page screenshots of the operation pages of each task node in the first task, so that the user can subsequently know the task content corresponding to each task node by retrieving these page screenshots, improving the operation flexibility and human-computer interaction of the electronic device.

[0182] In some embodiments of the present application, the electronic device can execute the above-mentioned first task in the above-mentioned virtual screen and take a screenshot of the operation page of the task node in the virtual screen, so that the third-party application can run in the virtual screen without adapting to the small window mode, saving development manpower and reducing the development cost of the application in the electronic device.

[0183] In some embodiments of the present application, the relevant information of the task node of the first task may include the node identifier of the task node that the first task has currently completed. After step 203, the information display method provided by the embodiments of the present application may include the following steps 401 and 402.

[0184] Step 401: The electronic device receives a first input from the user for the first node identifier among the displayed node identifiers.

[0185] Wherein, the above-mentioned first node identifier is the node identifier among the node identifiers of the task node that the first task has currently completed.

[0186] In some embodiments of the present application, the above first input may include, but is not limited to: a touch input of the first node identifier by a user through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. Specifically, it can be determined according to actual usage requirements, and the embodiments of the present application do not make limitations.

[0187] In some embodiments of the present application, the above touch input may be a click input, a long - press input, a press - and - hold input, or any possible touch input, and the embodiments of the present application do not make limitations. Among them, the click input may be a single - click input, a double - click input, or a click input of any number of times.

[0188] In some embodiments of the present application, the above specific gesture may be any one of a single - click gesture, a swipe gesture, a drag gesture, a pressure - recognition gesture, a long - press gesture, an area - change gesture, a double - press gesture, and a double - click gesture.

[0189] Step 402: In response to the first input, the electronic device displays a first screenshot in the first area.

[0190] Among them, the above first screenshot is a screenshot of the operation page of the task node corresponding to the first node identifier in the above virtual screen.

[0191] In some embodiments of the present application, the user can trigger the electronic device to display the above first screenshot in the above first area through the first input to the above first node identifier.

[0192] It can be understood that the above first screenshot can also be displayed in the first area.

[0193] In some embodiments of the present application, after the electronic device receives the above first input, the electronic device can, based on the association relationship between the task node corresponding to the above first node identifier and the image identifier of the above first screenshot, retrieve the stored above first screenshot, and after scaling the above first screenshot in proportion, display the scaled - down first screenshot in the above first area.

[0194] For example, after the user clicks the first node identifier, the electronic device can reduce the first screenshot in equal proportion and display the equally scaled - down first screenshot in the first area.

[0195] It should be noted that the above example is an example of equal - proportion reduction. In actual implementation, the electronic device can reduce or enlarge the first image in any proportion, and the embodiments of the present application do not make limitations in this regard.

[0196] Next, with reference to the accompanying drawings, the above steps 401 and 402 are exemplarily described.

[0197] Exemplarily, in combination with the above Scenario 1, assume that the application used for the coffee purchase task is Mini Program AA, and the task node corresponding to the first node identifier is "Enter Mini Program AA". Then, after the user clicks on the task node identifier 45 as shown in Figure 4 , the electronic device can, based on the association relationship between the task node identifier 45 and the image identifier of the page screenshot of the operation page of "Enter Mini Program AA", retrieve the stored page screenshot of the operation page of "Enter Mini Program AA", and reduce the page screenshot proportionally, and display it in the first area 60 as shown in Figure 6 : The page screenshot 62 of the operation page of "Enter Mini Program AA" after being reduced proportionally.

[0198] In this way, the user can input the above first node identifier to trigger the electronic device to display the first screenshot in the first area. Thus, the user can, by inputting the node identifier, trigger the electronic device to display the image content of the screenshot, know the page information of a single task node of the first task, and improve the information display effect of the electronic device.

[0199] Furthermore, the application for executing the first task can be without the need to adapt to the display ratio of the small window in advance, which can not only save the development manpower of the application, but also ensure the functional integrity of the application, avoid the problem of function loss when the application adapts to the small window. Therefore, it can improve the efficiency of the electronic device in executing tasks and save the development cost of the application.

[0200] In some embodiments of the present application, the interaction components included in the first area may include a second control. After the above step 203, the information display method provided by the embodiments of the present application may include the following steps 501 and 502.

[0201] Step 501: The electronic device receives a third input from the user to the second control.

[0202] It can be understood that the first area may include interaction components. Among them, the interaction components may include a second control.

[0203] In some embodiments of the present application, the above second control may be used to trigger the electronic device to execute the above first task in the foreground.

[0204] In some embodiments of the present application, the above third input may include, but is not limited to: the user's touch input to the second control through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. It can be specifically determined according to the actual usage requirements, and the embodiments of the present application do not make limitations.

[0205] In some embodiments of the present application, the above touch input may be a click input, a long - press input, a hard - press input, or any possible touch input, and the embodiments of the present application do not make any limitations. Among them, the click input may be a single - click input, a double - click input, or a click input of any number of times.

[0206] In some embodiments of the present application, the above - mentioned specific gesture may be any one of a single - click gesture, a swipe gesture, a drag gesture, a pressure - recognition gesture, a long - press gesture, an area - change gesture, a double - press gesture, and a double - click gesture.

[0207] Step 502: The electronic device responds to the third input and executes the first task in the foreground.

[0208] In some embodiments of the present application, executing the first task in the foreground can be understood as: The electronic device executes the first task on the screen, and the user can view the execution progress of the first task in real time through the screen of the electronic device.

[0209] In some embodiments of the present application, the user can trigger the electronic device to execute the above - mentioned first task in the foreground by inputting to the above - mentioned second control.

[0210] The following will exemplarily illustrate the above - mentioned steps 501 and 502 in conjunction with the accompanying drawings.

[0211] Exemplarily, in combination with the above - mentioned scenario 1, assume that the first task is a coffee - buying task. The user can click on the foreground execution 71 (i.e., the second control) in the interaction component 70 as shown in Figure 7A to trigger the electronic device to display the page 72 as shown in Figure 7B on the foreground. Among them, the page 72 is the operation page of the next task node to be executed after the task node of "input 'coconut latte'". That is to say, the electronic device executes the coffee - buying task in the foreground.

[0212] In this way, the user can trigger the electronic device to execute the first task in the foreground through the third input to the second control, so that the user can intuitively view the specific execution process of the first task, improving the human - machine interaction of the electronic device and the flexibility of the displayed information.

[0213] In some embodiments of the present application, the interaction components included in the above - mentioned first area may include a first control. After the above - mentioned step 203, the information - display method provided by the embodiments of the present application may include the following steps 601 and 602.

[0214] Step 601: The electronic device receives a second input from the user to the first control.

[0215] In some embodiments of the present application, the above-mentioned first control can be used to trigger the electronic device to display, on the foreground, a screenshot of the operation page of the last task node among the task nodes where the above-mentioned first task has been completed.

[0216] In some embodiments of the present application, the above-mentioned second input may include, but is not limited to: a touch input by the user on the first control using a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs. It can be specifically determined according to actual usage requirements, and the embodiments of the present application do not make any limitations.

[0217] In some embodiments of the present application, the above-mentioned touch input may be a click input, a long press input, a hard press input, or any possible touch input, and the embodiments of the present application do not make any limitations. Among them, the click input may be a single click input, a double click input, or a click input of any number of times.

[0218] In some embodiments of the present application, the above-mentioned specific gesture may be any one of a single click gesture, a swipe gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double press gesture, and a double click gesture.

[0219] Step 602: The electronic device responds to the second input and displays a second screenshot.

[0220] Among them, the above-mentioned second screenshot is a screenshot of the operation page of the last task node among the task nodes where the first task in the above-mentioned virtual screen has been currently completed.

[0221] In some embodiments of the present application, displaying the second screenshot can be understood as: The electronic device displays, on the screen, a screenshot of the operation page of the most recently completed task node, and the user can view the execution progress of the first task through the second screenshot displayed on the screen of the electronic device.

[0222] In some embodiments of the present application, after the user performs a second input on the above-mentioned first control, the electronic device can display the above-mentioned second screenshot on the screen. When the electronic device completes the execution action corresponding to the next task node in the above-mentioned virtual screen, the electronic device can update the second screenshot displayed on the screen to a screenshot of the operation page of the next task node.

[0223] In some embodiments of the present application, after the user performs the above-mentioned second input on the above-mentioned first control, the electronic device can, based on the association relationship between the task node and the image identifier of the screenshot of the operation page of the task node, retrieve the stored screenshot of the operation page of the task node and display the screenshot on the screen.

[0224] In some embodiments of the present application, the electronic device may scale the above-mentioned second screenshot and then display it on the screen of the electronic device. That is to say, the above-mentioned second screenshot displayed on the electronic device is an image obtained by scaling the above-mentioned second screenshot by the electronic device.

[0225] It should be noted that the above embodiments are described by taking the electronic device retrieving the page screenshot of the operation page of the stored task node as an example. In actual implementation, the electronic device may also send the page screenshot of the operation page of the task node captured on the above-mentioned virtual screen to the screen of the electronic device or the interaction component in the first area. The embodiments of the present application do not limit this.

[0226] In some embodiments of the present application, when the electronic device displays the above-mentioned second screenshot, the user can perform any operation on the electronic device.

[0227] In some embodiments of the present application, the above-mentioned second screenshot can be used as a display function in the electronic device. After the user inputs to the second screenshot, the electronic device operation virtual screen cannot be triggered.

[0228] Next, with reference to the accompanying drawings, the above steps 601 and 602 will be described by way of example.

[0229] Exemplarily, in combination with the above scenario 1, assume that the first task is a coffee purchase task. The user can click on the foreground reduction display 73 (i.e., the first control) in the interaction component 70 as shown in Figure 7C to trigger the electronic device to display the page screenshot 74 as shown in Figure 7D on the foreground. Among them, the page screenshot 74 is the operation page after the execution action of the next task node of the task node "input 'coconut latte'" is completed.

[0230] In this way, the user can trigger the electronic device to display the second screenshot on the current screen through the second input to the first control, so that the user can know the screenshot of the detailed page of the node in the execution process of the first task through the second screenshot, thereby adapting to the viewing preferences of different users for task progress notifications and improving the human-computer interaction and display information flexibility of the electronic device.

[0231] Next, with reference to the accompanying drawings, the information display method provided by the embodiments of the present application will be described by way of example.

[0232] Exemplarily, as shown in Figure 8 , assume that the first area includes the interaction component "Atomic Island". The information display method provided by the embodiments of the present application may include the following steps 11 to 16.

[0233] Step 11: The user creates a task in the electronic device through natural language and clarifies the requirements through the AI assistant A in the electronic device.

[0234] In this example, the above step 11 can be specifically implemented by the following step 11A and step 11B.

[0235] Step 11A: The user triggers the electronic device to start the AI assistant through voice input, and the user inputs instructions in the AI assistant using natural language.

[0236] For example, the user can input in the AI assistant: "Help me order a large cup of AA's coconut latte, iced, without extra sugar."

[0237] Step 11B: The AI assistant analyzes the instructions input by the user, pops up a confirmation box after analysis, displays the parameters corresponding to the task to be executed, and confirms the requirements with the user. If the user clicks to purchase, the task starts to be executed; if the user clicks to modify, the user can modify the current requirements.

[0238] Step 12: The electronic device starts the virtual screen and projects the current page of the electronic device. The electronic device starts the virtual screen in the background and ensures that the page information on the virtual screen is consistent with the current foreground page information of the electronic device.

[0239] Step 13: The electronic device obtains the current virtual screen information, and the AI assistant generates an automatic execution action.

[0240] In this example, the above step 13 can be implemented by the following steps 13A to 13C.

[0241] Step 13A: The electronic device takes a screenshot of the current virtual screen page, obtains the captured image, and obtains the virtual screen XML information through the accessibility interface of the electronic device.

[0242] Step 13B: The electronic device combines the picture (i.e., the captured image in step 13A) and the XML information into a prompt. The result of the prompt is as follows:

[0243] You are an automatic operation assistant (i.e., the AI assistant) in an electronic device. You need to judge the specific actions of the next automatic operation according to the task objective and the current page information.

[0244] Current task objective: aaaaaa

[0245] Current page XML information: bbbbbb

[0246] Current page picture: cccccc

[0247] Please give the next action and action description:

[0248] Step 13C: The electronic device requests the end-side large model with the prompt, and the large model returns the automatic operation action and action description.

[0249] Among them, the action description is mainly in text form, concisely summarizing the automatic operation actions returned by the large model. For example, the action description can be: "Click the search box", "Click the return button", "Enter 'coconut latte'", "Return to the previous menu", etc.

[0250] Step 14: The virtual screen executes the automatic operation actions generated by the large model.

[0251] In this example, the above Step 14 can be specifically implemented through the following Step 14A.

[0252] Step 14A: The electronic device executes the corresponding actions on the virtual screen according to the action type and corresponding parameters generated by the large model.

[0253] For example, if the action type output by the large model is input, and the action parameter is {'area': [100, 100, 150, 150], 'text': 'latte'}, it means that the electronic device needs to input "latte" in the area with coordinates [100, 100, 150, 150] on the virtual screen.

[0254] Step 15: After the automatic operation actions are executed, the electronic device takes a screenshot of the virtual screen and sends the action description and the screenshot back to the Atomic Island.

[0255] Step 16: On the Atomic Island, all completed steps are displayed in the form of nodes until the task ends.

[0256] In this example, the above Step 16 can be implemented through the following Step 16A or 16B or 16C or 16D.

[0257] Step 16A: By default, when the automatic operation task is in progress, the Atomic Island will be displayed in a reduced form on the current screen, only showing the APP icon and the fixed words that the current electronic device is operating on the virtual screen.

[0258] Step 16B: When the user clicks on the Atomic Island, the Atomic Island will be enlarged and display the steps that have been completed in the current task in the form of nodes. The node judges and displays the action description; when the user clicks on the node, the electronic device will display the corresponding step screenshot on the current screen at a scaled ratio.

[0259] Step 16C: When the user clicks the [Execute in Foreground] button on the Atomic Island, the task execution mode will be switched from the background form to the foreground form, and the user can observe the task execution status in real time on the current screen.

[0260] Step 16D: When the user clicks the [Reduce and Display in Foreground] button on the Atomic Island, each step screenshot will be displayed on the current mobile phone screen as an equally scaled-down picture.

[0261] Thus, the information display method provided by the embodiments of the present application can not only enable third-party applications to be adapted to the small window mode without additional effort, saving development manpower, but also allow users to monitor the task progress of the current task in real time and understand the page where the current task is being executed, so as to ensure that users are aware of the risk of personal privacy leakage. Moreover, when users experience the convenience of automatic operation of the electronic device, they can freely operate the electronic device, saving waiting time. In this way, the ability of the electronic device to execute tasks in parallel and the human-computer interaction performance can be improved.

[0262] For the information display method provided by the embodiments of the present application, the execution entity may be an information display device. In the embodiments of the present application, taking the information display device executing the information display method as an example, the information display device provided by the embodiments of the present application is described.

[0263] Combined with Figure 9 , an information display device 900 is provided by the embodiments of the present application. The information display device 900 may include a receiving module 901, a processing module 902, and a display module 903. The receiving module 901 may be configured to receive a task description prompt word input by a user, and the task description prompt word may include task description information of a first task. The processing module 902 may be configured to execute the first task based on the task description prompt word received by the receiving module 901. The display module 903 may be configured to display at least one of the following in a first area: task progress description information for describing the current task progress of the first task, an application identifier of an application program currently executing the first task, and relevant information of a task node that the first task has currently completed.

[0264] In some embodiments of the present application, the above information display device may further include: a startup module. The startup module may be configured to start a virtual screen in the background after the receiving module 901 receives the task description prompt word input by the user. Specifically, the processing module 902 may be configured to execute the first task in the virtual screen started by the startup module based on the task description prompt word.

[0265] In some embodiments of the present application, the relevant information of the task node of the first task may include a node identifier of the task node that the first task has currently completed. The receiving module 901 may further be configured to receive a first input of a user on a first node identifier among the displayed node identifiers after the display module 903 displays the relevant information of the task node that the first task has currently completed in the first area. The display module 903 may further be configured to, in response to the first input received by the receiving module 901, display a first screenshot in the first area, where the first screenshot is a page screenshot of an operation page of the task node corresponding to the first node identifier in the virtual screen.

[0266] In some embodiments of the present application, the interaction components included in the above-mentioned first region may include a first control. The above-mentioned receiving module 901 may also be used to receive a second input from the user to the first control; the above-mentioned display module 903 may also be used to display a second screenshot in response to the second input received by the above-mentioned receiving module 901, where the second screenshot is a screenshot of the operation page of the last task node among the currently completed task nodes of the first task in the above-mentioned virtual screen.

[0267] In some embodiments of the present application, the above-mentioned information display device may further include: a storage module. The above-mentioned processing module 902 may also be used to capture screenshots of the operation pages of each task node in the above-mentioned first task during the execution of the above-mentioned first task in the above-mentioned virtual screen after the virtual screen is started in the background by the above-mentioned start module; the storage module may be used to store the screenshots of the operation pages of each task node obtained by the above-mentioned processing module 902.

[0268] In the information display device provided in the embodiments of the present application, after receiving a task description prompt word input by the user, the information display device can analyze the task description prompt word to execute the task indicated by the task description prompt word, and display the task-related information of the task in the first region, so that the execution progress of the task can be viewed in real time without affecting the user's operation of the information display device, improving the ability of the information display device to execute tasks in parallel and the human-computer interaction.

[0269] The information display device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than a terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0270] The information display device in the embodiments of the present application can be a device with an operating system. The operating system can be the Android operating system, the iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

[0271] The information display device provided in the embodiments of the present application can implement each process implemented in the above method embodiments, achieving the same technical effects. To avoid repetition, details are not described herein again.

[0272] Optionally, as Figure 10 shown, the embodiments of the present application further provide an electronic device 800, including a processor 801 and a memory 802. A program or instruction that can run on the processor 801 is stored on the memory 802. When the program or instruction is executed by the processor 801, each step of the above information display method embodiment is implemented, and the same technical effects can be achieved. To avoid repetition, details are not described herein again.

[0273] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0274] Figure 11 It is a schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0275] The electronic device 100 includes, but is not limited to: a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110, and other components.

[0276] Those skilled in the art can understand that the electronic device 100 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 110 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 11 The structure of the electronic device shown in does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which are not described herein again.

[0277] Among them, the user input unit 107 can be used to receive a task description prompt word input by the user, and the task description prompt word can include the task description information of the first task. The processor 110 can be used to execute the first task based on the task description prompt word received by the user input unit 107. The display unit 106 can be used to display at least one of the following in the first area: task progress description information for describing the current task progress of the first task, an application identifier of the application currently executing the first task, and relevant information about the task nodes that have been completed for the first task currently.

[0278] In some embodiments of the present application, the processor 110 can be used to start a virtual screen in the background after the user input unit 107 receives the task description prompt word input by the user; specifically, the processor 110 can be used to execute the first task in the virtual screen started by the processor 110 based on the task description prompt word.

[0279] In some embodiments of the present application, the relevant information about the task nodes of the first task can include the node identifier of the task nodes that have been completed for the first task currently. The user input unit 107 can also be used to receive a first input from the user to the first node identifier among the displayed node identifiers after the display unit 106 displays the relevant information about the task nodes that have been completed for the first task currently in the first area; the display unit 106 can also be used to display a first screenshot in the first area in response to the first input received by the user input unit 107, and the first screenshot is a page screenshot of the operation page of the task node corresponding to the first node identifier in the virtual screen.

[0280] In some embodiments of the present application, a first control can be included in the interaction components included in the first area. The user input unit 107 can also be used to receive a second input from the user to the first control; the display unit 106 can also be used to display a second screenshot in response to the second input received by the user input unit 107, and the second screenshot is a page screenshot of the operation page of the last task node among the task nodes that have been completed for the first task currently in the virtual screen.

[0281] In some embodiments of the present application, the processor 110 can also be used to capture a page screenshot of the operation page of each task node in the first task during the execution of the first task in the virtual screen after the processor 110 starts the virtual screen in the background; the memory 109 can be used to store the page screenshots of the operation pages of each task node obtained by the processor 110.

[0282] In the electronic device provided in the embodiment of the present application, after receiving the task description prompt word input by the user, the electronic device can analyze the task description prompt word to execute the task indicated by the task description prompt word, and display the task-related information of the task in the first area, so that the execution progress of the task can be viewed in real time without affecting the user's operation of the electronic device, improving the parallel task execution ability and human-computer interaction of the electronic device.

[0283] It should be understood that in the embodiment of the present application, the input unit 104 may include a Graphics Processing Unit (GPU) 1041 and a microphone 1042. The graphics processor 1041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. The other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0284] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 109 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0285] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 110 either.

[0286] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above information display method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0287] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0288] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above information display method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0289] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0290] The embodiments of the present application provide a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above information display method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0291] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0292] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0293] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. An information display method, characterized in that, The method includes: Receiving a task description prompt word input by a user, where the task description prompt word includes task description information of a first task; Executing the first task based on the task description prompt word; Displaying at least one of the following in a first area: task progress description information for describing the current task progress of the first task, an application identifier of the application currently executing the first task, and relevant information of the task nodes that the first task has currently completed.

2. The method according to claim 1, wherein After receiving the task description prompt word input by the user, the method further includes: Starting a virtual screen in the background; The executing the first task based on the task description prompt word includes: Based on the task description prompt word, executing the first task in the virtual screen.

3. The method according to claim 2, wherein The relevant information of the task node includes the node identifier of the task node that the first task has currently completed; After displaying the relevant information of the task nodes that the first task has currently completed in the first area, the method further includes: Receiving a first input from the user for a first node identifier among the displayed node identifiers; In response to the first input, displaying a first screenshot in the first area, where the first screenshot is a page screenshot of the operation page of the task node corresponding to the first node identifier in the virtual screen.

4. The method according to claim 2, wherein A first control is included in the interaction components included in the first area; the method further includes: Receiving a second input from the user for the first control; In response to the second input, displaying a second screenshot, where the second screenshot is a page screenshot of the operation page of the last task node among the task nodes that the first task has currently completed in the virtual screen.

5. The method according to any one of claims 2 to 4, characterized in that After starting the virtual screen in the background, the method further includes: During the process of executing the first task in the virtual screen, taking a page screenshot of the operation page of each task node in the first task to obtain a page screenshot of the operation page of each task node; Storing the page screenshots of the operation pages of each task node.

6. An information display device, characterized in that, The device includes: a receiving module, a processing module, and a display module; The receiving module is configured to receive a task description prompt word input by a user, where the task description prompt word includes task description information of a first task; The processing module is configured to execute the first task based on the task description prompt word received by the receiving module; The display module is configured to display at least one of the following in a first area: task progress description information for describing the current task progress of the first task, an application identifier of the application currently executing the first task, and relevant information of the task nodes that the first task has currently completed.

7. The device according to claim 6, characterized in that, The device further includes: a starting module; The starting module is configured to start a virtual screen in the background after the receiving module receives a task description prompt word input by the user; The processing module is specifically configured to execute the first task in the virtual screen started by the starting module based on the task description prompt word.

8. The device according to claim 7, characterized in that The relevant information of the task node includes the node identifier of the task node that the first task has currently completed; The receiving module is further configured to, after the display module displays the relevant information of the task nodes that have been completed for the first task in the first area, receive a first input from the user for a first node identifier among the displayed node identifiers; The display module is further configured to, in response to the first input received by the receiving module, display a first screenshot in the first area, where the first screenshot is a page screenshot of the operation page of the task node corresponding to the first node identifier in the virtual screen.

9. The device according to claim 7, characterized in that, The first area includes an interaction component that contains a first control; The receiving module is further configured to receive a second input from the user for the first control; The display module is further configured to, in response to the second input received by the receiving module, display a second screenshot, where the second screenshot is a page screenshot of the operation page of the last task node among the task nodes that have been completed for the first task in the virtual screen.

10. The device according to claims 6 to 9, characterized in that, The device further includes: a storage module; The processing module is further configured to, after the startup module starts the virtual screen in the background, during the execution of the first task in the virtual screen, capture screenshots of the operation pages of each task node in the first task to obtain page screenshots of the operation pages of each task node; The storage module is configured to store the page screenshots of the operation pages of each task node obtained by the processing module.

Images