Task execution method and device, electronic equipment, medium and program product

Abstract

The invention discloses a task execution method and device, electronic equipment, a medium and a program product, and belongs to the technical field of terminals. The method comprises the steps of determining intention information of a user according to first input of the user, wherein the intention information is used for representing a target task to be executed in a first application; according to the intention information and interface information of M interfaces in the first application, M pieces of operation information corresponding to the M interfaces in a one-to-one mode are predicted, and M is a positive integer; according to the M pieces of operation information, operation is executed on the M interfaces.

Description

Technical Field

[0001] The present application belongs to the field of terminal technology, and specifically relates to a task execution method, device, electronic device, medium and program product. Background Art

[0002] When using electronic devices, such as mobile phones, users often need to perform tedious and repetitive operations.

[0003] For example, when a user wants to order the takeaway that he or she ordered last time, the user needs to first click on the icon of the takeaway application to trigger the mobile phone to display the main interface of the takeaway application, and then click on the personal center in the main interface to find the historical order interface, and then find the required order link in the historical order interface, so that the user can input the order link to trigger the electronic device to order the takeaway corresponding to the order link again.

[0004] In this way, the user needs to perform a series of operations before the electronic device can complete the user's target task, that is, the operation efficiency of the user to achieve the target task through the electronic device is low. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a task execution method, device, electronic device, medium and program product, which can improve the operational efficiency of electronic devices in achieving user target tasks.

[0006] In a first aspect, an embodiment of the present application provides a task execution method, the method comprising: determining the user's intention information based on the user's first input, the intention information being used to characterize a target task to be executed in a first application; predicting M operation information corresponding one-to-one to the M interfaces based on the intention information and interface information of M interfaces in the first application, where M is a positive integer; and performing operations on the M interfaces respectively based on the M operation information.

[0007] In the second aspect, an embodiment of the present application provides a task execution device, which may include: the determination module, used to determine the user's intention information based on the user's first input, and the intention information is used to characterize the target task to be executed in the first application; the prediction module, used to predict M operation information corresponding to the M interfaces in the first application based on the interface information of the M interfaces and the intention information determined by the determination module, where M is a positive integer; the execution module, used to perform operations on the M interfaces respectively according to the M operation information predicted by the prediction module.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In the embodiment of the present application, since the intention information characterizing the target task to be performed by the user in the first application can be determined based on the user's first input, and the operation information corresponding to the M interfaces can be predicted based on the intention information and the interface information of the M interfaces in the first application, it can be ensured that the M interfaces and the M operation information are the same as the interfaces and operation information required for the user to manually trigger the first application to perform the target task, thereby ensuring that the first application can perform the target task after simulating the operation on the M interfaces according to the M operation information. In this way, after determining the user's intention information based on the user's first input, the target task can be automatically performed through the first application by operation, thereby simplifying and improving the operational efficiency of the electronic device in performing the target task. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flowchart of a task execution method provided in an embodiment of the present application;

[0014] Figure 2A This is a schematic diagram of an interface for applying a task execution method provided in an embodiment of the present application;

[0015] Figure 2B This is a schematic diagram of an interface for applying a task execution method provided in an embodiment of the present application;

[0016] Figure 2C This is a schematic diagram of an interface for applying a task execution method provided in an embodiment of the present application;

[0017] Figure 3AThis is a schematic diagram of an interface for applying a task execution method provided in an embodiment of the present application;

[0018] Figure 3B This is a schematic diagram of an interface for applying a task execution method provided in an embodiment of the present application;

[0019] Figure 4 It is a flowchart of a task execution method provided in an embodiment of the present application;

[0020] Figure 5 It is a flowchart of a task execution method provided in an embodiment of the present application;

[0021] Figure 6 It is a flowchart of a task execution method provided in an embodiment of the present application;

[0022] Figure 7 It is a structural diagram of a task execution device provided in an embodiment of the present application;

[0023] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0024] Fig. 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The task execution method, device, electronic device, medium and program product provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0028] The task execution method, device, electronic device, medium, and program product provided in the embodiments of the present application can be applied to scenarios where users use electronic devices.

[0029] For example, in a scenario where a user needs to use an electronic device to order food, listen to music, read the news, search for questions, or find coupons, or in a scenario where a user needs to use a function of the electronic device that the user uses less frequently or has never used, the task execution method provided in the embodiments of the present application can be used to simplify the user operation process and operation difficulty and improve the operating efficiency of the electronic device.

[0030] The task execution method provided in the embodiment of the present application can be executed by an electronic device or at least one of a functional module and an entity module in the electronic device that can implement the task execution method. The specific execution can be determined according to actual use requirements and is not limited in the embodiment of the present application. In the embodiment of the present application, the task execution method provided in the embodiment of the present application is described by taking the execution of the task execution method by an electronic device as an example.

[0031] The present application embodiment provides a task execution method, such as Figure 1 As shown, the task execution method provided in the embodiment of the present application may include the following steps:

[0032] Step 101: The electronic device determines user intention information according to a first input of the user.

[0033] The intent information may be used to characterize the target task to be performed by the user in the first application.

[0034] In some embodiments of the present application, the first application may be any possible application installed in the electronic device, which is specifically determined according to the first information.

[0035] For example, the first application may be a music application, a map application, a video application, a food delivery application, a social application, etc.

[0036] In some embodiments of the present application, the target task may be any possible task of the first application, which is specifically determined according to the first information.

[0037] For example, the target task may be: ordering takeout through the first application, playing a certain music, playing a certain video, searching for a certain place, searching for a certain keyword, etc.

[0038] In some embodiments of the present application, the first input may include any one of the following (1) and (2):

[0039] (1) Select the input of the first information.

[0040] (2) inputting first information into the operation control, where the operation control can be used to trigger a simulated operation function of the electronic device;

[0041] The above-mentioned intention information may be determined based on the first information. For example, the above-mentioned intention information may be determined based on the semantics of the first information.

[0042] In some embodiments of the present application, in the above (1), the first input may be an input of a user to first information displayed by the electronic device, wherein the first information may include any of the following: text, image, audio or video.

[0043] In some embodiments of the present application, each text may include any of the following: a sentence, a paragraph, a word.

[0044] In some embodiments of the present application, in the above (1), the first input may be a drag input or a copy input of the first information by the user.

[0045] For example, the electronic device displays at least one text, and the user can drag one text in the at least one text, so that the electronic device can determine the user's intention information based on the text.

[0046] For example, the electronic device displays at least one image, and the user may copy one of the at least one image, so that the electronic device may determine the user's intention information based on the image.

[0047] In some embodiments of the present application, in the above (2), the user can input voice or text in the operation control.

[0048] For example, the user can enter text in the operation control: "Help me place an order for zz mobile phone in a app."

[0049] In some embodiments of the present application, the operation control may be an information input box.

[0050] In some embodiments of the present application, the operation control may be located in a taskbar interface, a negative one-screen interface, a desktop, or any other possible interface.

[0051] In this way, since the user triggers the electronic device to determine the first information by selection or manual input, the flexibility of determining the first information can be improved.

[0052] In some embodiments of the present application, the electronic device may determine the intention information according to the semantics of the first information. Specifically, after the electronic device obtains the first information, it may first obtain the semantics of the first information and determine the intention information according to the semantics.

[0053] In some embodiments of the present application, when the first information is text, the electronic device can perform semantic recognition on the first information and determine the intention information based on the recognized semantics. When the first information is an image, the electronic device can first perform image recognition on the image to obtain at least one of the semantics 1 of the text information in the image and the semantics 2 expressed by the image object in the image, and then the electronic device can determine the intention information based on the recognized semantics.

[0054] In some embodiments of the present application, when the first information is an image, the electronic device may recognize text information in the image by using optical character recognition (OCR) technology.

[0055] In some embodiments of the present application, if the electronic device determines the intention information according to the semantics of the first information, the electronic device can display a confirmation message to facilitate the user to confirm whether to enable the simulation operation function, and the user can trigger the electronic device to enable the simulation operation function of the electronic device by confirming the confirmation message, so that the electronic device can continue to perform the following step 102. Of course, the user can also trigger the electronic device to stop enabling the simulation operation function by canceling the confirmation message.

[0056] For example, Figure 2A As shown, the electronic device displays a conversation interface 20, and the user can long press on a conversation message in the conversation interface 20: "Help me place an order for yy mobile phone in application a", that is, the user's first input, triggering the electronic device to copy the conversation message 21, so that the electronic device can determine the user's intention information as "place an order for yy mobile phone in application a" according to the semantics of the conversation message, and then Figure 2B As shown, the electronic device displays prompt information 2223. If the user inputs the "yes" option in prompt information 22, that is, confirms the input, the electronic device can continue to execute step 102. Otherwise, the electronic device cancels the display of prompt information 22 and ends the process.

[0057] For example, Figure 3A As shown, the electronic device displays a social interface 30, which includes a promotional image 31 of "aa documentary". If the user wants to watch "aa documentary", the user can slide on the promotional image 31, that is, the first input of the user, to trigger the electronic device to perform image recognition on the promotional image 31, and determine the user's intention information according to the recognition result, such as determining that the user's intention information is: need to play "aa documentary" through the video application; then Figure 3B As shown, the electronic device displays prompt information 32. If the user inputs the "yes" option in the prompt information 32, that is, confirms the input, the electronic device can continue to execute step 102. Otherwise, the electronic device cancels the display of the prompt information 32 and ends the process.

[0058] In some embodiments of the present application, the above step 101 can be implemented by the following step 101A.

[0059] Step 101A: The electronic device determines the user's intention information according to the semantics of the first information and the first corresponding relationship.

[0060] The first corresponding relationship is the corresponding relationship between applications and application categories in the electronic device.

[0061] For the first information being information selected or input through the first input, refer to (1) and (2) above.

[0062] It can be understood that the user's intention information can be determined more quickly and accurately based on the semantics of the first information and the first corresponding relationship.

[0063] In some embodiments of the present application, application categories may include but are not limited to: work, music, shopping, takeout, travel, system application, social, game, video, call, navigation, life, etc.

[0064] In some embodiments of the present application, each application category in the first corresponding relationship may correspond to at least one application. For example, video 1 and video 2 both correspond to the video category.

[0065] In some embodiments of the present application, the first application relationship may be a default one for the electronic device or may be set by the user, which is not limited in the present application.

[0066] In some embodiments of the present application, the electronic device may first determine the task that the user requires the electronic device to perform based on the semantics of the first information, then determine the first application based on the type of task and the first correspondence, and then determine the user's intention information based on the task and the first application.

[0067] Exemplarily, assuming that the first information is: "Order the dumplings ordered last time", the electronic device can determine that the semantics of the first information is "Order the dumplings ordered last time in the takeaway application", so that the electronic device can determine that the task the user needs the electronic device to perform is "Order dumplings", and the task type of "Order dumplings" is "Takeaway", so that the electronic device can find application a whose application type is takeaway from the electronic device based on "Takeaway" and the first correspondence, and determine that the user's intention information includes: "Order the dumplings bought last time through application a", so as to represent the task of ordering the dumplings bought last time to be executed in application a through the intention information.

[0068] In some embodiments of the present application, the electronic device may input the first information into the second model, and dynamically determine the application that the user needs to operate through the second model. The prompt instruction of the second model may be as follows: You are a user intention judgment assistant, and you need to judge whether the user request is an operation intention, what application to use, and how to start the application based on the semantics of the first information and the first corresponding relationship.

[0069] For example, the first information includes: "Play Zhou Helun's songs". The electronic device can input the first information into the second model, and the second model performs semantic analysis on the first information to determine that the user needs to listen to "Zhou Helun's" songs. If in the first correspondence, application 2 is a music application, the second model can output a judgment result: "true#music#application2#fullscreen". Among them, full screen refers to the interface display mode of application 2 when application 2 plays Zhou Helun's music. The display mode can be a default display mode or determined by the semantics of the first information.

[0070] In some embodiments of the present application, the second model may be a neural network model.

[0071] In some embodiments of the present application, when the electronic device determines multiple applications according to the target task, the electronic device can determine an intention information according to each application and the target task to obtain multiple intention information, so that the electronic device can sequentially perform the following step 102 for the multiple intention information. Of course, the electronic device can output prompt information, and the user selects one intention information from the multiple intention information to perform step 102.

[0072] For example, assuming the target task is "listen to Zhou Hye Lun's songs", and application 2 and application 3 in the electronic device are both music applications, then the electronic device can determine intent information 1 "play Zhou Hye Lun's songs with application 3" and intent information 2 "play Zhou Hye Lun's songs with application 3".

[0073] In this way, since the user's intention information can be determined based on the semantics of the first information determined by the first input and the first corresponding relationship, the user's intention can be judged more quickly and accurately, thereby improving the accuracy of determining the intention information.

[0074] Step 102: The electronic device predicts M operation information corresponding to the M interfaces one by one based on the intention information and the interface information of the M interfaces in the first application.

[0075] Wherein, M can be a positive integer.

[0076] It can be understood that the above M interfaces may include each interface operated by the user during the process of the user manually triggering the first application to execute the target task. The M operation information may be used to indicate the operation path of the user manually triggering the first application to execute the target task.

[0077] In some embodiments of the present application, the M interfaces may be different interfaces in the first application. It should be noted that in the present embodiment, different interface states of the same interface in the first application are regarded as different interfaces in the M interfaces.

[0078] For example, Figure 2A As shown in FIG. 1 , a conversation interface 20 in a conversation application is shown. If a user clicks an input box 24 in the conversation interface 20, Figure 2C As shown, the electronic device can display the input method interface 25 on the conversation interface 20. Then: Figure 2A The displayed session interface 20 and Figure 2C The interface 26 including the conversation interface 20 and the input method interface 25 shown may be referred to as different interfaces in the first application.

[0079] In some embodiments of the present application, the interface information of the interface may include component parameters of each operating component in the interface.

[0080] In some embodiments of the present application, the operating components in the interface may include but are not limited to at least one of the following: operating controls, operating identifiers, input boxes, operating options, and any other components that can perform human-computer interaction with the user.

[0081] In some embodiments of the present application, the above-mentioned component parameters may include: component size, component location information, component name, etc.

[0082] The component location information may be: the location information of the component in the interface when the electronic device displays the interface to which the component belongs; or the component location information may be: the location information of the component in the screen of the electronic device when the electronic device displays the interface to which the component belongs.

[0083] It should be noted that the interface information of the interface may also include at least one of the following: an identifier of the first application, an interface identifier, and a level of the interface.

[0084] In some embodiments of the present application, the operation information may include at least one of the following: information of the operation object, the operation type, the operation position, and the operation content. It should be noted that the operation object may be an operation component.

[0085] The operation type may include but is not limited to any of the following: click, double-click, long press, slide, drag, perform a specific gesture, input a specific voice, and input a specific text.

[0086] The operation object may be an operation component displayed by the electronic device. For example, the operation object may be a navigation key, a music play button, a shopping link, a video play link, a takeaway link, an interface switching control, etc. displayed by the electronic device.

[0087] In some embodiments of the present application, the information of the operation object can be represented as: the interface identifier of the interface where the operation object is located and the identifier of the operation object. In this way, the execution operation can be simulated more accurately according to the operation information. Among them, the interface identifier may include at least one of the following: the label of the interface, the name of the interface, and the interface level, and the identifier of the operation object may include any of the following: the name of the operation object, the image of the operation object, and the content of the operation object.

[0088] For example, the operation object can be expressed as: "yy component in a interface" or "a interface-yy component".

[0089] In some embodiments of the present application, each piece of operation information may indicate an operation.

[0090] In some embodiments of the present application, the electronic device can simulate the user performing the operation indicated by the operation information based on the operation information.

[0091] For example, if the operation information indicates a click operation on component 1 , the electronic device may activate component 1 .

[0092] In some embodiments of the present application, the i+1th interface among the M interfaces mentioned above can be determined based on the user's intention information and the i-th interface among the M interfaces, and the value range of i is an integer less than M; for each interface among the M interfaces, the above step 102 can be implemented through the following step 102A.

[0093] Step 102A: The electronic device inputs the intention information and the interface information of the ith interface into the first model for operation prediction processing, and obtains the ith operation information corresponding to the ith interface output by the first model.

[0094] It can be understood that the model parameters of the first model can be obtained by training based on the operation information of manually triggering multiple types of applications to perform various tasks and the interface corresponding to the operation information. In this way, since the model parameters of the first model are obtained by training based on the operation information of manually triggering multiple types of applications to perform various tasks and the interface corresponding to the operation information, the first model can accurately predict the operation information of the operation that needs to be performed on the i-th interface in order to achieve the target task based on the intention information and the interface information of the i-th interface, thereby improving the accuracy of automatically performing operations on the i-th interface.

[0095] In some embodiments of the present application, the model parameters of the first model can be specifically trained based on the operation information of manually triggering multiple types of applications to perform various tasks and the interface information of the interface corresponding to each operation information. Thus, the first model can predict the user's operation on the i-th interface based on the intention information and the interface information of the i-th interface: during the triggering process of the user manually triggering the first application to perform the target task, the user's operation on the i-th interface.

[0096] In some embodiments of the present application, after the electronic device inputs the intent information and the interface information of the ith interface into the first model, the first model can first determine at least one operation information of the user manually triggering the first application to perform the target task based on the intent information, each operation information corresponds to an interface in the first application, and then the first model can select the operation information that matches the ith interface in the at least one operation information, hereinafter referred to as candidate operation information, and calibrate the candidate operation information with the interface information of the ith interface, and use the calibrated operation information as the ith operation information corresponding to the ith interface. Among them, the operation information matching the ith interface can be: the interface identifier corresponding to the operation information is the same as the interface identifier of the ith interface.

[0097] For example, the first model can calibrate the operation position, operation object, operation area, etc. in the above candidate operation information according to the interface information of the i-th interface, so that the calibrated operation information matches the display parameters of the i-th interface displayed by the electronic device.

[0098] In some embodiments of the present application, the second model may be the same as or different from the first model.

[0099] In this way, it can be ensured that the operation indicated by the operation information matches the display parameters of the i-th interface displayed by the electronic device, thereby improving the accuracy of the simulated operation sequence and simulated operation position of the electronic device performing the operation, and further improving the reliability and success rate of the simulated operation.

[0100] In some embodiments of the present application, the above step 102A can be implemented through the following step 102A1.

[0101] Step 102A1: The electronic device inputs the intention information, the interface information of the i-th interface and the historical operation information into the first model for operation prediction processing, and obtains the i-th operation information corresponding to the i-th interface output by the first model.

[0102] The historical operation information may at least indicate: the operations automatically performed on the first application for the most recent N times after the electronic device determined the user's intention information, where N is a positive integer.

[0103] For example, assuming that when a user manually triggers the first application to perform a target task, 10 operations need to be performed on the first application, and before entering the i-th interface, the electronic device has simulated the first 3 operations of the 10 operations, then:

[0104] (1) The electronic device can input the intention information, the interface information of the i-th interface and the operation information when simulating the third operation into the operation prediction model for operation prediction processing to obtain the i-th operation information corresponding to the i-th interface output by the first model.

[0105] Alternatively, (2) the electronic device may input the intention information, the interface information of the i-th interface and the operation information of the first three operations into the first model for operation prediction processing to obtain the i-th operation information corresponding to the i-th interface output by the first model.

[0106] Here, i = 4. Also, N = 1 in (1) and N = 3 in (2).

[0107] In some embodiments of the present application, the electronic device may also input the operation result of the historical operation information into the first model to assist the first model in predicting the i-th operation information. It is understood that the operation result may be the operation result interface corresponding to each operation information in the historical operation information or the interface information of the operation result interface.

[0108] In this way, since the historical operation information can indicate the simulated trigger progress of automatically triggering the first application to execute the target application, the i-th operation information corresponding to the i-th interface can be more accurately predicted based on the historical operation information, the operation intention and the interface information of the i-th interface.

[0109] In this way, since the i+1th interface among the M interfaces is determined by the first model based on the intention information and the i-th interface among the M interfaces, that is, the latter interface is determined comprehensively based on the user's intention information and the previous interface, it can be ensured that the M interfaces are the interfaces operated when the user manually triggers the first application to perform the target task, and predicting the next interface through the first model can provide prediction accuracy and prediction speed, thereby improving the response speed of the electronic device.

[0110] Step 103: The electronic device performs operations on the M interfaces respectively according to the M operation information.

[0111] After performing simulation operations on the M interfaces, the first application executes the target task.

[0112] In some embodiments of the present application, for each piece of operation information among the M operation information, the electronic device performs the operation indicated by the operation information on the interface corresponding to the operation information according to the operation information.

[0113] In some embodiments of the present application, an electronic device performs an operation indicated by an operation information on an interface corresponding to the operation information, which may include: the electronic device activates an operation object indicated by the operation information in the interface according to the operation information, to operate on the operation object.

[0114] In some embodiments of the present application, after the electronic device performs an operation on an interface, the electronic device may display an interface corresponding to the operation in the first application. In other words, after the electronic device performs an operation on each of the M interfaces, the electronic device may display an interface in the M interfaces that is adjacent to each interface and located behind the one interface.

[0115] It should be noted that the electronic device display interface can be any of the following: an interface displayed in the background of the electronic device that is invisible to the user; an interface displayed in the foreground of the electronic device that is visible to the user.

[0116] It can be understood that the task execution method provided in the embodiment of the present application can simulate the completion of the user's target task step by step based on various needs determined by the user's first input (i.e., the first information) and the terminal side environmental information of the electronic device, i.e., the interface of the first application displayed by the electronic device, without relying on external interfaces, and has high versatility and convenience.

[0117] The task execution method provided in the embodiment of the present application is exemplified below with reference to specific examples.

[0118] For example, it is assumed that the user manually triggers application 2 to play music A, which includes four operations, namely:

[0119] Step 1: Click the icon of application A to trigger the main interface of A music;

[0120] Step 2: Click the search box in the main interface to bring up the input method interface;

[0121] Step 3: Enter the text "A music" in the search box on the main interface to enter the search information;

[0122] Step 4: Click "A Music Option" to trigger application A to play A music.

[0123] If the electronic device has currently executed the second step, the electronic device may input the following information into the first model:

[0124] a1: Historical operation information: "Step 1: click on the icon of music A, Step 2: click on the search box";

[0125] b1: interface information of the main interface including the input method in application 2;

[0126] c1: Intent message "Please use application 2 to play music A".

[0127] Then, the first model can select the third operation information of the operation to be performed in step 3 based on the above information. The third operation information can indicate: click on the letters contained in the pinyin of "A Music" in the input method interface, so that the electronic device can simulate entering "A Music" in the input box according to the operation information.

[0128] After the electronic device performs the third operation corresponding to the third operation information on the input method interface, the electronic device may input the following information into the first model:

[0129] a2: Historical operation information: "Step 1: Click on the icon of A Music, Step 2: Click on the search box, Step 3: Enter the text "A Music" in the search box on the main interface"

[0130] b2: interface information of the search result interface obtained by searching "A music" by application 2;

[0131] c2: Intent message "Please use application 2 to play music A".

[0132] Therefore, the first model can select the 4th operation information of the operation to be performed in step 4 according to a2, b2 and c2, and the 4th operation information can indicate: click on "the option of music A". Then, the electronic device can simulate clicking on the option of music A in the search result interface according to the 4th operation information, so that the electronic device can play music A through the first application. It should be noted that the operation information of the above-mentioned "first step" is directly determined based on the intention information, that is, the M operation information in step 102 does not include the operation information of the "first step" in this example.

[0133] In this way, after receiving the user's first input to determine the first information "play music through application A", the electronic device can perform a series of operations to automatically trigger application 2 to play music A without the need for manual operation by the user, thereby simplifying the operation process of the electronic device and improving operating efficiency.

[0134] In the task execution method provided in the embodiment of the present application, since the intention information characterizing the target task to be executed by the user in the first application can be determined based on the user's first input, and the operation information corresponding to the M interfaces can be predicted based on the intention information and the interface information of the M interfaces in the first application, it can be ensured that the M interfaces and the M operation information are the same as the interfaces and operation information required for the user to manually trigger the first application to execute the target task, thereby ensuring that the first application can execute the target task after simulating the execution of operations on the M interfaces based on the M operation information. In this way, since the target task can be automatically executed through the first application by simulating the user's operation after the user's intention information is determined based on the user's first input, the operation efficiency of the electronic device in executing the target task can be simplified and improved.

[0135] In some embodiments of the present application, the above step 103 may include the following step 103A. After step 103A, the task execution method provided by the embodiment of the present application may further include the following step 104.

[0136] Step 103a: The electronic device performs an i-th operation corresponding to the i-th operation information on the i-th interface according to the i-th operation information.

[0137] Step 104: In response to the i-th operation, the electronic device displays an operation result interface corresponding to the i-th operation in the first application.

[0138] The operation result interface may be the (i+1)th interface among the above-mentioned M interfaces.

[0139] It can be understood that after the electronic device performs an operation on the i-th interface among the M interfaces, it can trigger the electronic device to display the i+1-th interface among the M interfaces.

[0140] In some embodiments of the present application, the electronic device may display the above-mentioned operation result interface in any of the following ways: full screen, small window, split screen, and background.

[0141] In some embodiments of the present application, the display mode of the operation result interface may be a default mode of the electronic device, or may be determined based on the intention information.

[0142] For example, if the intent information is: "Application 2 plays A's music in the background", the electronic device can display A's music playback interface in application 2 in the background of the electronic device after performing M operations on application 2 according to M operation information, so as to play A's music through application 2.

[0143] In this way, after the electronic device automatically performs an operation, the electronic device can display an operation result interface of the operation, so that the user can confirm the progress of the simulation or automatic operation according to the operation result interface.

[0144] In some embodiments of the present application, before the above step 102A, the task execution method provided by the embodiment of the present application may further include the following step 105.

[0145] Step 105: The electronic device performs information calibration processing on the interface information of the i-th interface using the screenshot image as a reference.

[0146] It should be noted that the electronic device can input the interface information of the ith interface after calibration into the first model.

[0147] In some embodiments of the present application, the electronic device may obtain Extensible Markup Language (XML) interface information of the ith interface while displaying the ith interface, and obtain a screenshot image of the ith interface.

[0148] Among them, the electronic device can obtain the XML interface information of the ith interface from the interface data of the ith interface.

[0149] It should be noted that the XML interface information of the interface is determined by the display parameters of the interface, and the size of the screenshot image of the interface is the same as the display size of the interface.

[0150] In some embodiments of the present application, after the electronic device obtains the XML interface information and screenshot image of the i-th interface, it can parse the screenshot image, obtain the parameters of the text information and marking information in the screenshot image, and obtain the reference information of the i-th interface; wherein each marking information indicates an operating component in the i-th interface, and the parameters of each marking information may include: the display size, color, shape, etc. of the operating component indicated by the marking information.

[0151] In addition, the electronic device can simplify the XML interface information of the ith interface, delete the background call parameters, the color of the operating components, the interface level, etc. in the XML interface information, retain the position information and size information of the operating components in the ith interface, and obtain the component parameters of the operating components. Then, the electronic device can use the reference information of the ith interface as a reference to align the component parameters of each operating component in the interface information of the ith interface with the corresponding marking information in the reference information. Among them, the process of aligning the component parameters of each operating component in the interface information of the ith interface with the corresponding marking information in the reference information can also be called the end-side information enhancement process.

[0152] For example, if the position information 1 of an operating component in the interface information is different from the position information 2 in the mark information indicating the operating component in the reference information, the position information of the operating component in the interface information is modified to the position information 2;

[0153] If the interface information includes component parameters of the operating component 2, but the reference information does not include tag information indicating the operating component 2, the electronic device may delete the component parameters of the operating component 2 from the interface information;

[0154] If the interface information does not include component parameters of component 3, and the reference information includes tag information indicating the operation of component 3, the electronic device can add the tag information indicating the operation of component 3 in the reference information to the interface information as the component parameters of the operation of component 3.

[0155] It can be understood that the interface information of the i-th interface can indicate the layout between the operating components in the i-th interface.

[0156] In this way, since the screenshot image of the i-th interface can be used to calibrate the interface information of the i-th interface, the accuracy of the interface information can be improved, thereby improving the accuracy of the i-th operation information determined by the first model, and further improving the success rate of the electronic device automatically triggering the first application to perform the target task.

[0157] The complete process of the task execution method provided in the embodiment of the present application is described below.

[0158] For example, Figure 4 As shown, the task execution method provided in the embodiment of the present application may include the following steps:

[0159] Step 401: The electronic device senses and analyzes the user request.

[0160] It should be noted that the electronic device can perceive the user request by receiving the user's first input. It can be understood that the purpose of the electronic device parsing the user request is to determine the user's intention information.

[0161] Step 402: The electronic device executes a terminal-side action and obtains terminal-side information.

[0162] In some embodiments of the present application, the electronic device performing the terminal-side operation may include any of the following:

[0163] The electronic device performs an operation on the icon of the first application, and displays the main interface of the first application in response to the operation;

[0164] The electronic device performs an i-th operation on the i-th interface among M interfaces, and displays the i+1-th interface among the M interfaces in response to the i-th operation, where i and M are positive integers, and the value range of i is a positive integer less than M.

[0165] Among them, the M interfaces are different interfaces in the first application, and the first interface among the M interfaces is the main interface of the first application, and M can be a positive integer.

[0166] In some embodiments of the present application, the electronic device acquiring the terminal side information may include: acquiring XML interface information and a screenshot image of the interface currently displayed by the electronic device in the first application.

[0167] Step 403: The electronic device parses the terminal-side information.

[0168] It can be understood that the method for the electronic device to parse the terminal side information refers to the relevant description of step 105 in the above embodiment.

[0169] Step 404: The electronic device outputs operation information for the next operation through the first model according to the intention information and the terminal side information obtained by the analysis.

[0170] For example, the electronic device outputs the i-th operation information corresponding to the i-th interface through the first model based on the intention information and the interface information of the i-th interface.

[0171] Step 405: The electronic device determines whether the simulation operation process is completed.

[0172] In some embodiments of the present application, if the electronic device determines that the simulation operation process has not ended, the electronic device may repeatedly perform step 102 until the simulation operation process is determined to be ended, and perform step 406 after the simulation operation process.

[0173] Step 406: The electronic device executes the target task through the first application.

[0174] In some embodiments of the present application, if the electronic device has started to execute the target task, it is determined that the simulated operation process has ended, otherwise it is determined that the simulated operation process has not ended.

[0175] In some embodiments of the present application, Figure 4 ,like Figure 5 As shown, step 401 may be implemented through steps A1 and A2, or may be implemented through steps B1, B2 and B3, or may be implemented through steps C1 and C2.

[0176] Step A1: The electronic device receives a user's input of dragging or copying text.

[0177] Step A2: The electronic device determines the user's intention information based on the text.

[0178] Step B1: The electronic device receives a user's input of dragging or copying an image.

[0179] Step B2: The electronic device extracts a plurality of first text information from the image according to an OCR algorithm.

[0180] Step B3: The electronic device determines the user's intention information based on the multiple first text messages.

[0181] Step C1: The electronic device receives a user input of a plurality of second text information in an operation control.

[0182] Step C2: The electronic device determines the user's intention information based on the multiple second text messages.

[0183] Among them, the above-mentioned text, image and multiple second text information are all the first information in the above-mentioned embodiment.

[0184] For other descriptions of steps A1, A2, B1, B2, B3, C1 and C2, refer to the relevant descriptions in the above embodiments.

[0185] In some embodiments of the present application, Figure 5 ,like Figure 6 As shown, before the above step 102, the task execution method provided in the embodiment of the present application may further include the following step 407.

[0186] Step 407: The electronic device determines whether to initiate a simulation operation process.

[0187] It can be understood that after the electronic device determines the user's intention information, it can output a prompt message, and the user determines whether to trigger the simulation operation process. If the user triggers the electronic device to initiate the simulation operation process through input, the electronic device can continue to execute step 402, otherwise the electronic device ends the execution of the simulation operation process.

[0188] In the task execution method provided in the embodiment of the present application, since it is possible to simulate step by step the task that the user needs the electronic device to perform based on the user's first input and the terminal-side environmental information, i.e., the interface information of the first application displayed on the electronic device, there is no need to rely on an external interface, and the method has high versatility and convenience.

[0189] The task execution method provided in the embodiment of the present application can be executed by a task execution device. In the embodiment of the present application, the task execution device provided in the embodiment of the present application is described by taking the task execution method executed by the task execution device as an example.

[0190] The present application embodiment provides a task execution device, Figure 7 The structure diagram of the task execution device provided in the embodiment of the present application is shown as follows: Figure 7 As shown, the task execution device 700 provided in the embodiment of the present application may include: a determination module 701, a prediction module 702 and an execution module 703;

[0191] The determination module 701 may be used to determine the user's intention information according to the user's first input, where the intention information is used to represent the target task to be executed in the first application;

[0192] The prediction module 702 may be used to predict M operation information corresponding to the M interfaces one by one according to the interface information of the M interfaces in the first application and the intention information determined by the determination module 701, where M is a positive integer.

[0193] The execution module 703 may be used to respectively execute operations on the M interfaces according to the M pieces of operation information predicted by the prediction module 702 .

[0194] In some embodiments of the present application, the i+1th interface among the above-mentioned M interfaces is determined based on the above-mentioned intention information and the i-th interface among the M interfaces, and the value range of i is an integer less than M; the determination module 701 can be specifically used to input the intention information and the interface information of the i-th interface into the first model for operation prediction processing, and obtain the i-th operation information corresponding to the i-th interface output by the first model.

[0195] In some embodiments of the present application, the prediction module 702 may be specifically used to input the intention information, the interface information of the i-th interface, and the historical operation information into the first model for operation prediction processing to obtain the i-th operation information output by the first model;

[0196] The historical operation information at least indicates: the operations performed on the first application for the most recent N times after the intention information is determined.

[0197] In some embodiments of the present application, the task execution device may further include a processing module; the processing module may be used to input the intention information and the interface information of the i-th interface into the first model for operation prediction processing in the prediction module, and before obtaining the i-th operation information corresponding to the i-th interface output by the first model, use the screenshot image of the i-th interface as a reference to perform information calibration processing on the interface information of the i-th interface.

[0198] In some embodiments of the present application, the prediction module may be specifically used to perform an i-th operation corresponding to the i-th operation information on the i-th interface according to the i-th operation information;

[0199] The above-mentioned task execution device may also include a display module; the display module can be used to display the operation result interface corresponding to the i-th operation in the first application in response to the above-mentioned i-th operation performed by the above-mentioned prediction module; wherein the operation result interface can be the i+1th interface among the above-mentioned M interfaces.

[0200] In some embodiments of the present application, the above-mentioned determination module may be specifically used to determine the intention information according to the semantics of the first information and the first corresponding relationship, and the first corresponding relationship may be a corresponding relationship between an application and an application category in the task execution device;

[0201] The first input may include any of the following:

[0202] selecting input of the first information;

[0203] The first information is input into an operation control, and the operation control can be used to trigger a simulation operation function of a task execution device.

[0204] In the task execution device provided in the embodiment of the present application, since the task execution device can determine the intention information representing the target task to be executed by the user in the first application according to the first input of the user, and predict the operation information corresponding to the M interfaces one by one according to the intention information and the interface information of the M interfaces in the first application, it can ensure that the M interfaces and the M operation information are the same as the interfaces and operation information required for the user to manually trigger the first application to execute the target task, thereby ensuring that the first application can execute the target task after simulating the execution of operations on the M interfaces according to the M operation information. In this way, since the target task can be automatically executed through the first application by operation after the user's intention information is determined according to the first input of the user, the operational efficiency of the task execution device in executing the target task can be simplified and improved.

[0205] The task execution device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0206] The task execution device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0207] The task execution device provided in the embodiment of the present application can achieve Figures 1 to 6 The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.

[0208] Alternatively, if Figure 8As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned task execution method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0209] It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.

[0210] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0211] The electronic device 1500 includes but is not limited to: a radio frequency unit 1501, a network module 1502, an audio output unit 1503, an input unit 1504, a sensor 1505, a display unit 1506, a user input unit 1507, an interface unit 1508, a memory 1509, and a processor 1510 and other components.

[0212] Those skilled in the art will appreciate that the electronic device 1500 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1510 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0213] The processor 1510 may be used to determine the user's intention information according to the user's first input, and the intention information may be used to characterize the target task to be performed in the first application;

[0214] Processor 1510 may be configured to predict M pieces of operation information corresponding to the M interfaces one-to-one based on interface information of the M interfaces in the first application and the intention information, where M is a positive integer.

[0215] The processor 1510 may be configured to respectively perform operations on the M interfaces according to the M operation information.

[0216] In some embodiments of the present application, the i+1th interface among the above-mentioned M interfaces is determined based on the above-mentioned intention information and the i-th interface among the above-mentioned M interfaces, and the value range of i is an integer less than M; the above-mentioned processor 1510 can be specifically used to input the above-mentioned intention information and the interface information of the i-th interface into the first model for operation prediction processing, and obtain the i-th operation information corresponding to the i-th interface output by the first model.

[0217] In some embodiments of the present application, the processor 1510 may be specifically configured to input the intention information, the interface information of the i-th interface, and the historical operation information into the first model for operation prediction processing, and obtain the i-th operation information output by the first model;

[0218] The historical operation information at least indicates: the operations performed on the first application for the most recent N times after the intention information is determined.

[0219] In some embodiments of the present application, the processor 1510 may be used to perform information calibration processing on the interface information of the i-th interface using a screenshot of the i-th interface as a reference before inputting the intention information and the interface information of the i-th interface into the first model for operation prediction processing and obtaining the i-th operation information corresponding to the i-th interface output by the first model.

[0220] In some embodiments of the present application, the processor 1510 may be specifically configured to perform an i-th operation corresponding to the i-th operation information on the i-th interface according to the i-th operation information;

[0221] The display unit 1506 may be configured to display an operation result interface corresponding to the i-th operation in the first application in response to the i-th operation performed by the processor 1510;

[0222] Among them, the above-mentioned operation result interface can be the i+1th interface among the above-mentioned M interfaces.

[0223] In some embodiments of the present application, the processor 1510 may be specifically used to determine the intention information according to the semantics of the first information and the first correspondence relationship, where the first correspondence relationship may be a correspondence relationship between an application and an application category in a task execution device.

[0224] The first input may include any of the following:

[0225] Select the input of the first information;

[0226] The first information is input into the operation control, and the operation control can be used to trigger the simulation operation function of the task execution device.

[0227] In the electronic device provided by the embodiment of the present application, since the electronic device can determine the intention information representing the target task to be performed by the user in the first application based on the user's first input, and predict the operation information corresponding to the M interfaces one by one based on the intention information and the interface information of the M interfaces in the first application, it can ensure that the M interfaces and the M operation information are the same as the interfaces and operation information required for the user to manually trigger the first application to perform the target task, thereby ensuring that the first application can perform the target task after simulating the operation on the M interfaces according to the M operation information. In this way, since the target task can be automatically performed through the first application by operation after the user's intention information is determined according to the user's first input, the operation efficiency of the electronic device in performing the target task can be simplified and improved.

[0228] It should be understood that in the embodiment of the present application, the input unit 1504 may include a graphics processor (Graphics Processing Unit, GPU) 15041 and a microphone 15042, and the graphics processor 15041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1506 may include a display panel 15061, and the display panel 15061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1507 includes a touch panel 15071 and at least one of other input devices 15072. The touch panel 15071 is also called a touch screen. The touch panel 15071 may include two parts: a touch detection device and a touch controller. Other input devices 15072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0229] The memory 1509 can be used to store software programs and various data. The memory 1509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1509 may include a volatile memory or a non-volatile memory, or the memory 1509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0230] The processor 1510 may include one or more processing units; optionally, the processor 1510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1510.

[0231] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned task execution method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0232] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0233] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned task execution method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0234] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0235] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned task execution method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0236] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A task execution method, characterized in that: The method comprises: Determining, according to a first input of the user, intention information of the user, wherein the intention information is used to represent a target task to be performed in the first application; According to the intention information and the interface information of the M interfaces in the first application, predict M operation information corresponding to the M interfaces one by one, where M is a positive integer; According to the M pieces of operation information, operations are performed on the M interfaces respectively.

2. The method according to claim 1, characterized in that The i+1th interface among the M interfaces is determined according to the intention information and the ith interface among the M interfaces, and the value range of i is an integer less than M; The predicting, based on the intention information and the interface information of the M interfaces in the first application, M operation information corresponding to the M interfaces one by one, includes: The intention information and the interface information of the i-th interface are input into the first model for operation prediction processing to obtain the i-th operation information corresponding to the i-th interface output by the first model.

3. The method according to claim 2, characterized in that The step of inputting the intention information and the interface information of the i-th interface into a first model for operation prediction processing to obtain the i-th operation information corresponding to the i-th interface output by the first model includes: Inputting the intention information, the interface information of the i-th interface and the historical operation information into the first model for operation prediction processing to obtain the i-th operation information; The historical operation information at least indicates: the operations performed on the first application for the most recent N times after the intention information is determined, where N is a positive integer.

4. The method according to claim 2 or 3, characterized in that: The performing operations on the M interfaces respectively according to the M operation information includes: According to the i-th operation information, perform an i-th operation corresponding to the i-th operation information on the i-th interface; The method further comprises: In response to the i-th operation, displaying an operation result interface corresponding to the i-th operation in the first application; Among them, the operation result interface is the (i+1)th interface among the M interfaces.

5. The method according to claim 1, characterized in that The determining the user's intention information according to the user's first input includes: Determining the intention information according to the semantics of the first information and a first correspondence relationship, wherein the first correspondence relationship is a correspondence relationship between applications and application categories in the electronic device; The first input includes any one of the following: selecting the input of the first information; inputting the first information in an operation control, and the operation control is used to trigger a simulation operation function of the electronic device.

6. A task execution device, characterized in that: The device comprises: a determination module, a prediction module and an execution module; The determination module is used to determine the user's intention information according to the user's first input, where the intention information is used to represent the target task to be executed in the first application; The prediction module is used to predict M operation information corresponding to the M interfaces one by one according to the interface information of the M interfaces in the first application and the intention information determined by the determination module, where M is a positive integer; The execution module is used to respectively execute operations on the M interfaces according to the M operation information predicted by the prediction module.

7. The device according to claim 6, characterized in that The i+1th interface among the M interfaces is determined according to the intention information and the ith interface among the M interfaces, and the value range of i is an integer less than M; The determination module is specifically used to input the intention information and the interface information of the i-th interface into the first model for operation prediction processing, and obtain the i-th operation information corresponding to the i-th interface output by the first model.

8. The device according to claim 7, characterized in that The prediction module is specifically used to input the intention information, the interface information of the i-th interface and the historical operation information into the first model for operation prediction processing, and obtain the i-th operation information output by the first model; The historical operation information at least indicates: the operations performed on the first application for the most recent N times after the intention information is determined, where N is a positive integer.

9. The device according to claim 7 or 8, characterized in that The execution module is specifically configured to execute, on the i-th interface, an i-th operation corresponding to the i-th operation information according to the i-th operation information; the device further comprises a display module; The display module is configured to display an operation result interface corresponding to the i-th operation in the first application in response to the i-th operation performed by the execution module; Among them, the operation result interface is the (i+1)th interface among the M interfaces.

10. The device according to claim 6, characterized in that The determination module is specifically configured to determine the intention information according to the semantics of the first information and a first corresponding relationship, wherein the first corresponding relationship is a corresponding relationship between applications and application categories in the electronic device; The user input includes any one of the following: selecting the input of the first information; inputting the first information in an operation control, and the operation control is used to trigger a simulation operation function of the electronic device.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the task execution method according to any one of claims 1 to 5 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the task execution method according to any one of claims 1 to 5 are implemented.

13. A computer program product, characterized in that The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the task execution method according to any one of claims 1 to 5.

Images