Cross-end task processing method and device, electronic equipment, medium and program product

Abstract

The application discloses a cross-terminal task processing method and device, electronic equipment, medium and program product. The method comprises the following steps: obtaining task indication information corresponding to a to-be-processed task, wherein the task indication information is conversation information input by a current account on a first terminal and directed to a preset intelligent agent; performing task analysis on the task indication information based on the preset intelligent agent to obtain a task instruction sequence, wherein the task instruction sequence is an operation instruction in a process of executing the to-be-processed task based on a target application, and the target application is an application program corresponding to a second terminal; deploying the target application in a local emulator instance to obtain an emulator instance carrying the target application; and executing the task instruction sequence based on the emulator instance carrying the target application. The technical scheme provided by the application can improve the task processing efficiency and convenience, effectively cope with different cross-terminal task processing requirements, and improve the efficiency and interaction convenience of cross-terminal collaborative processing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a cross-platform task processing method, apparatus, electronic device, medium, and program product. Background Technology

[0002] Currently, cross-platform technologies are frequently used in scenarios such as office automation. While cross-platform technologies such as remote control software, multi-screen collaborative projection software, and PC (Personal Computer) automation tools can solve the problem of cumbersome operation caused by repeatedly switching devices to some extent, the former two often require users to perform repeated manual operations, resulting in low efficiency in cross-platform collaborative processing. The latter can make tools repeatedly execute predetermined tasks by writing scripts or recording operation processes, but scripts cannot be directly reused, resulting in high operation and time costs, and efficiency is still limited. Summary of the Invention

[0003] This application provides a cross-platform task processing method, apparatus, device, storage medium, and computer program product, which can improve task processing efficiency and convenience, effectively address different cross-platform task processing needs, greatly reduce the operation and time costs of cross-platform task processing, and improve the efficiency and interactive convenience of cross-platform collaborative processing.

[0004] On the one hand, this application provides a cross-platform task processing method, the method comprising: Obtain task instruction information corresponding to the task to be processed. The task instruction information is the dialogue information entered by the current account on the first terminal for a preset intelligent agent. Based on the preset intelligent agent, the task instruction information is parsed to obtain a task instruction sequence. The task instruction sequence is at least one task operation instruction in the process of the target application executing the task to be processed. The target application is the application corresponding to the second terminal. Deploy the target application in a local emulator instance to obtain an emulator instance that hosts the target application; The task instruction sequence is executed based on the simulator instance that carries the target application.

[0005] On the other hand, a cross-platform task processing device is provided, the device comprising: The task instruction information acquisition module is configured to acquire the task instruction information corresponding to the task to be processed. The task instruction information is the dialogue information of the current account inputting on the first terminal for a preset intelligent agent. The task parsing module is configured to perform task parsing on the task indication information based on the preset intelligent agent to obtain a task instruction sequence. The task instruction sequence is at least one task operation instruction in the process of executing the task to be processed based on the target application. The target application is the application corresponding to the second terminal. The application deployment module is configured to deploy the target application in a local simulator instance to obtain a simulator instance that hosts the target application; The task execution module is configured to execute the task instruction sequence based on the simulator instance carrying the target application.

[0006] In an optional embodiment, the task parsing module includes: The semantic understanding unit is configured to perform semantic understanding of the task indication information based on the preset intelligent agent to obtain the task intent information corresponding to the task to be processed, the application identifier to be run corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed, wherein the application identifier to be run is the application identifier of the target application. The task decomposition unit is configured to input the task intent information, the application identifier to be run, and the first task execution parameters into the preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed. The instruction mapping unit is configured to perform instruction mapping based on the preset intelligent agent on the task operation chain information to obtain the task instruction sequence.

[0007] In an optional embodiment, the task decomposition unit is specifically configured to input the task intent information, the application identifier to be run, and the first task execution parameters into the preset agent, obtain the preset operation chain set corresponding to the target application based on the application identifier to be run, determine the matching operation chain information corresponding to the task to be processed from the preset operation chain set based on the task intent information, and generate task operation chain information based on the matching operation chain information and the first task execution parameters.

[0008] In an optional embodiment, the semantic understanding unit is further configured to perform semantic understanding of the task indication information based on the preset intelligent agent to obtain the task intent information, the application identifier to be run, the first task execution parameters, and parameter supplementation indication information; The parameter supplementation indication information is used to indicate the supplementary task execution parameters required to execute the task to be processed.

[0009] In an optional embodiment, the apparatus further includes: The first information display module is configured to display the parameter supplement prompt information in the dialogue interface between the current account and the preset intelligent agent. The parameter supplementation module is configured to respond to the parameter supplementation command and obtain the execution parameters for the second task. The task decomposition unit is further configured to decompose the task operation by inputting the task intent information, the application identifier to be run, the first task execution parameters and the second task execution parameters into the preset intelligent agent, thereby obtaining the task operation chain information.

[0010] In an optional embodiment, the apparatus further includes: The simulator instance determination module is configured to determine an idle simulator instance from a preset resource pool based on the preset agent; The application deployment module is also configured to deploy the target application in the local idle simulator instance to obtain a simulator instance carrying the target application.

[0011] In an optional embodiment, the task execution module is specifically configured to execute task instructions in the task instruction sequence sequentially based on the simulator instance carrying the target application, and to perform the following steps during the execution of any task operation instruction: Obtain the current interface control tree of the current application interface corresponding to any task operation instruction; the current interface control tree is used to indicate the layout of controls in the current application interface. Based on the current interface control tree, locate the operation control corresponding to any task operation instruction; Execute any of the task operation instructions based on the operation controls.

[0012] In an optional embodiment, the apparatus further includes: The information acquisition module is configured to acquire the current interface image corresponding to the target task instruction and the task execution log information corresponding to the task to be processed when the operation control corresponding to the target task instruction fails to be located. The target task instruction is any task operation instruction in the task instruction sequence whose operation control location fails to be located. The second information display module is configured to display the current interface image and the task execution log information in the dialogue interface between the current account and the preset intelligent agent.

[0013] In an optional embodiment, the apparatus further includes: The interface image acquisition module is configured to acquire the current interface image corresponding to the target task instruction when the operation control corresponding to the target task instruction fails to be located. The target task instruction is any task operation instruction in the task instruction sequence for which the operation control location fails to be located. The location information determination module is configured to perform control recognition on the current interface image and determine the location information of the operation control corresponding to the target task instruction in the current interface image. The control positioning module is configured to execute the operation control corresponding to the target task instruction based on the positioning information; The instruction execution module is configured to execute the target task instruction based on the operation control corresponding to the target task instruction.

[0014] In an optional embodiment, the simulator instance is provided with an automation executor, and the task execution module is further configured to execute the automation executor to control the target application in the simulator instance to automatically execute the task instruction sequence through the system interface of the simulator instance.

[0015] In an optional embodiment, the apparatus further includes: The execution progress display module is configured to display the task execution progress information corresponding to the task to be processed in the dialogue interface between the current account and the preset intelligent agent during the execution of the task instruction sequence. The task execution progress information includes at least one of the following: execution progress description text during the execution of the task instruction sequence, or execution interface images during the execution of the task instruction sequence.

[0016] In an optional embodiment, the apparatus further includes: The success indication information display module is configured to display the task success indication information corresponding to the task to be processed in the dialogue interface between the current account and the preset intelligent agent when the task to be processed is successfully executed. The task success indication information includes at least one of the following: task success message text, or the task completion interface image corresponding to the task to be processed in the target application.

[0017] On the other hand, an electronic device is provided, including: a processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the cross-platform task processing method described above.

[0018] On the other hand, a computer-readable storage medium is provided, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform any of the above-described cross-device task processing methods.

[0019] On the other hand, a computer program product or computer program is provided, the computer program product including instructions stored in a computer-readable storage medium, the processor retrieving the instructions from the computer-readable storage medium and executing the instructions to implement the cross-end task processing method provided in the various optional implementations described above.

[0020] The cross-platform task processing method, apparatus, device, storage medium, and computer program product provided in this application have the following technical effects: During the task execution process for the target application on the second terminal, task instruction information can be input on the first terminal via dialogue with a preset intelligent agent. This allows the preset intelligent agent to parse the task instruction information, resulting in operation instructions (task instruction sequence) for the target application to execute the task. This automatically parses the user's task instruction information into a specific operation sequence. The target application is then deployed in a local simulator instance, creating a simulator instance carrying the target application. Based on this simulator instance, the task instruction sequence is executed. This allows the application task execution on the second terminal to be triggered on the first terminal without requiring manual user operation of the application on the second terminal. No device switching is needed, significantly improving task processing efficiency and convenience. Furthermore, by integrating the intelligent task processing capabilities of the first terminal with the application execution capabilities of the second terminal, it effectively addresses different cross-terminal task processing needs, greatly reducing the operational and time costs of cross-terminal task processing and improving the efficiency and interactivity of cross-terminal collaborative processing. Attached Figure Description

[0021] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the application environment of a cross-platform task processing method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a cross-platform task processing method provided in an embodiment of this application; Figure 3 This is a schematic diagram of a dialogue interface between a current account and a preset intelligent agent, provided in an embodiment of this application. Figure 1; Figure 4 This is a schematic diagram of a dialogue interface between a current account and a preset intelligent agent, provided in an embodiment of this application. Figure 2 ; Figure 5 This is a schematic diagram of a dialogue interface between a current account and a preset intelligent agent, provided in an embodiment of this application. Figure 3 ; Figure 6 This is a schematic diagram of a dialog interface displaying parameter supplement prompts provided in an embodiment of this application; Figure 7 This is a schematic diagram of a dialog interface provided in this application embodiment to display task execution progress information corresponding to tasks to be processed. Figure 1 ; Figure 8 This is a schematic diagram of a dialog interface provided in this application embodiment to display task execution progress information corresponding to tasks to be processed. Figure 2 ; Figure 9 This is a schematic diagram of a dialog interface provided in this application embodiment to display task execution progress information corresponding to tasks to be processed. Figure 3 ; Figure 10 This is a schematic diagram of a dialog interface provided in this application embodiment, displaying a success indication message for a pending task. Figure 1 ; Figure 11 This is a schematic diagram of a dialog interface provided in this application embodiment, displaying a success indication message for a pending task. Figure 2 ; Figure 12 This is a schematic diagram of a dialog interface provided in this application embodiment, displaying a success indication message for a pending task. Figure 3 ; Figure 13 This is a schematic diagram of a cross-platform task processing procedure according to an exemplary embodiment. Figure 1 ; Figure 14 This is a schematic diagram of a cross-platform task processing procedure according to an exemplary embodiment. Figure 2 ; Figure 15 This is a schematic diagram of a cross-platform task processing procedure according to an exemplary embodiment. Figure 3 ; Figure 16 This is a schematic diagram of the structure of a cross-platform task processing device provided in an embodiment of this application; Figure 17 This is a block diagram of an electronic device for cross-platform task processing provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0025] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0026] The following section introduces some key terms or abbreviations involved in the embodiments of this application.

[0027] Large Language Models (LLMs) are a class of natural language processing models built using deep learning techniques, particularly neural networks. Through unsupervised or self-supervised learning on large-scale text corpora, they are able to understand, generate, and process natural language. With billions of parameters, LLMs possess powerful capabilities in language representation, knowledge retrieval, reasoning, and dialogue.

[0028] A pre-defined intelligent agent refers to an intelligent agent driven by a large language model. That is, an artificial intelligence system that uses a large language model as its core decision-making or interaction component, capable of autonomously perceiving the environment, understanding instructions, planning actions, and executing tasks; in other words, it is the subject that makes decisions and takes actions. Specifically, the pre-defined intelligent agent in this application embodiment can be used to parse task instruction information to obtain the corresponding task instruction sequence. Optionally, it can also be used to allocate simulator instances.

[0029] VDI (Virtual Disk Image) file: A VDI (Virtual Disk Image) file is a virtual disk image file. A VDI file can be viewed as a digital copy of a physical disk.

[0030] The emulator host refers to the emulator software (application) running on a computer, along with its associated hardware abstraction layer, operating system, and user interface components. The emulator host is responsible for creating and managing virtual clients (emulator guests) and provides a series of tools and interfaces so that users can interact with the virtual clients; its product form is an application capable of searching for and downloading various applications (such as a marketplace application).

[0031] The emulator guest client (emulator instance) refers to the virtual client (virtual client environment) created on a computer by the emulator software. In this environment, users can install and run cross-platform applications, performing various operations and experiences just like using a real cross-platform device; for example, installing and running a mobile application on a PC (mobile version application), or installing and running a PC version application (i.e., desktop application) on a mobile device. In this application embodiment, the emulator guest client can be a mobile or PC system implemented based on a VirtualBox (Virtual Box) virtualization scheme, and its representation is a VDI file.

[0032] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application. The application environment may include at least a first terminal 110 and a server 120.

[0033] The first terminal 110 and the server 120 are connected via a wireless or wired network. The first terminal 110 can be used to provide cross-platform task processing services to users; for example, it can handle at least one cross-platform task such as sending files for office work, booking TV series, booking flights, or ordering takeout for daily life services.

[0034] In an optional embodiment, the first terminal 110 may include an interaction module 111, a simulator management module 112, and a data exchange bus 113 (not shown in the figure); the server 120 may include a task agent engine 121, which is driven by a preset intelligent agent; specifically, the interaction module may be used to provide a graphical interactive interface, which may include a dialogue interface between the current account and the preset intelligent agent; and the graphical interactive interface is used to input task instruction information corresponding to the task to be processed, and can support at least one input method such as voice, text, and shortcut commands to input task instruction information corresponding to the task to be processed; the task to be processed may be a cross-terminal task to be processed, that is, a task executed based on the application corresponding to a non-local terminal (second terminal); optionally, the first terminal 110 may send the task instruction information input by the user through the interaction module to the server 120; optionally, the graphical interactive interface may also be used to display task execution progress information, task success indication information, etc. corresponding to the task to be processed.

[0035] In an optional embodiment, the server 120 can perform task parsing on the task indication information based on a preset intelligent agent (task agent engine 121) to obtain a sequence of task instructions during the execution of the task to be processed based on the target application. Specifically, the task agent engine 121 may include a semantic understanding unit 1211, a task decomposition unit 1212, and a simulator scheduling and mapping unit 1213. Specifically, the semantic understanding unit 1211 can be used to perform semantic understanding on the task indication information to determine the task intent information, the identifier of the application to be run (i.e., the application identifier of the target application to be run during the task processing), and the first task execution parameters. The task decomposition unit 1212 can be used to decompose the task operation based on the task intent information, the identifier of the application to be run, and the first task execution parameters to obtain the task operation chain information. The simulator scheduling and mapping unit 1213 can be used to manage the simulator resource pool (a preset resource pool, which may include simulator instance 1 and simulator instance 2), allocate simulator instances, and map the task operation chain information to the corresponding task instruction sequence. Specifically, the simulator management module 112 can be used to manage simulator instances on the first terminal. Specifically, it can deploy the application corresponding to the second terminal within the local simulator instance, obtaining a simulator instance carrying the corresponding application's requirements. Based on the simulator instance carrying the corresponding application, it executes a sequence of task instructions. During execution, the simulator instance can control the application within it to automatically execute the corresponding sequence of task instructions, thereby executing the pending tasks of the corresponding application on the second terminal on the first terminal, achieving cross-terminal task processing. Specifically, the data exchange bus 113 can be based on a preset communication protocol, responsible for the transmission of instructions and data between units; the preset communication protocol can include any of WebSocket (web socket), gRPC (g Remote Procedure Call framework), etc.

[0036] In an optional embodiment, the first terminal 110 can be a PC or a mobile terminal; if the first terminal 110 is a PC, the second terminal can be a mobile terminal and the target application is a mobile application; if the first terminal 110 is a mobile terminal, the second terminal can be a PC and the target application is a PC application.

[0037] In an optional embodiment, the first terminal 110 can be an electronic device running applications or other software; specifically, the application (client) on the first terminal 110 can create a simulator guest through the simulator host; the second terminal can be an electronic device running applications or other software. The application corresponding to the second terminal can be deployed on the first terminal 110 through a simulator instance.

[0038] In one specific embodiment, the electronic device may include, but is not limited to, smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, in-vehicle terminals, smart TVs, and other types of terminals, or it may be a server. Optionally, if the first terminal 110 is a PC, the operating system running on the first terminal 110 may include, but is not limited to, Windows, macOS, Linux, etc. Correspondingly, the emulator host may be PC software implemented based on a PC system such as Windows, and the product form is a PC application (i.e., a desktop application). Optionally, if the first terminal 110 is a mobile terminal, the operating system running on the first terminal 110 may include, but is not limited to, HarmonyOS, Android, iOS, etc. Correspondingly, the emulator host may be mobile software implemented based on a mobile system such as Android, and the product form is a mobile application.

[0039] In an optional embodiment, the server 120 can be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0040] In addition, it should be noted that, Figure 1 The example shown is merely one application environment provided by this disclosure. The embodiments of this application are not limited thereto. For example, the task agent engine 121 described above can also be deployed on the first terminal 110, or the task agent engine 121, the simulator management module 112 and the data exchange bus 113 can all be deployed on a server.

[0041] Those skilled in the art should understand that the first terminal 110 and server 120 described above are merely illustrative examples. Other existing or future terminals or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0042] The following describes a cross-platform task processing method proposed in this application. Figure 2 This is a flowchart illustrating a cross-platform task processing method provided in an embodiment of this application. This method can be executed by a first terminal, by a server, or jointly by both the first terminal and the server. The following example demonstrates execution by the first terminal. Figure 2 As shown, the method may include: S201: Obtain the task instruction information corresponding to the task to be processed.

[0043] In one specific embodiment, the aforementioned task instruction information is dialogue information input by the current account on the first terminal for a preset intelligent agent; the task to be processed can be a cross-terminal task to be processed, that is, a task executed based on the application corresponding to a non-local terminal; the non-local terminal can be a second terminal; for example, on the PC (first terminal), the dialogue information (task instruction information) input for a preset intelligent agent is executed based on the mobile application corresponding to the mobile terminal (second terminal); or on the mobile terminal (first terminal), the dialogue information (task instruction information) input for a preset intelligent agent is executed based on the PC application corresponding to the PC terminal (second terminal).

[0044] In an optional embodiment, the task to be processed may include any task such as cross-platform office tasks such as sending files, cross-platform life service tasks such as booking TV series, booking air tickets, and ordering takeout; specifically, the task instruction information may be information describing the task to be processed.

[0045] In one specific embodiment, the user can input task instructions through a dialogue interface between their current account and a preset intelligent agent. The current account can be the currently logged-in user account.

[0046] In a specific embodiment, taking a file sending task as an example, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of a dialogue interface between a current account and a preset intelligent agent, provided in an embodiment of this application. Figure 1 Specifically, in combination Figure 3 As can be seen, the task instruction information corresponding to the file sending task can be "Help me send the AA file on my desktop to BB through the XX application".

[0047] In a specific embodiment, taking the task to be processed as a TV series reservation task as an example, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of a dialogue interface between a current account and a preset intelligent agent, provided in an embodiment of this application. Figure 2 Specifically, in combination Figure 4 As can be seen, the task instruction information corresponding to the TV series reservation task can be "Help me reserve RRR Season 3 through the YY application".

[0048] In a specific embodiment, taking a flight booking task as an example, such as... Figure 5 As shown, Figure 5 This is a schematic diagram of a dialogue interface between a current account and a preset intelligent agent, provided in an embodiment of this application. Figure 3 Specifically, in combination Figure 4As can be seen, the task instruction information corresponding to the flight booking task can be "Help me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon through the ZZ app".

[0049] S203: Based on the preset intelligent agent, the task instruction information is parsed to obtain the task instruction sequence.

[0050] In one specific embodiment, the task instruction sequence can be at least one task operation instruction during the execution of a task to be processed by a target application; the target application can be an application corresponding to the second terminal. Specifically, the application corresponding to the second terminal can be an application running under the operating system environment of the second terminal.

[0051] In an optional embodiment, the above-mentioned task instruction sequence obtained by parsing the task indication information based on a preset intelligent agent may include: Based on the semantic understanding of the task instruction information by the preset intelligent agent, the task intent information, the application identifier to be run, and the first task execution parameters corresponding to the task to be processed are obtained. The application identifier to be run is the application identifier of the target application. Input the task intent information, the application identifier to be run, and the first task execution parameters into the preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed. Based on a pre-defined intelligent agent, the task operation chain information is mapped to obtain a task instruction sequence.

[0052] In a specific embodiment, task intent information can be used to indicate the task objective information corresponding to the task to be processed. For example, the task intent information (task objective) corresponding to the file sending task "Help me send the AA file on my desktop to BB through the XX application" can be sending a file; the task intent information (task objective) corresponding to the TV series reservation task "Help me reserve RRR Season 3 through the YY application" can be reserving a TV series. The task intent information (task objective) corresponding to the flight booking task "Help me book a flight from NN to MM around 15:00 on XX month XX day through the ZZ application" can be booking a flight.

[0053] In a specific embodiment, the application identifier to be run can be the identifier of the application (target application) required to run during the execution of the task to be processed, such as the application name or other information that can identify the application. For example, taking the file sending task "Help me send the AA file on my desktop to BB through the XX application" as an example, the application identifier to be run can be the identifier of the XX application; taking the TV series reservation task "Help me reserve RRR Season 3 through the YY application" as an example, the application identifier to be run can be the identifier of the YY application; taking the flight booking task "Help me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon" as an example, the application identifier to be run can be the ZZ application.

[0054] In a specific embodiment, the first task execution parameter corresponding to the task to be processed can be the parameters required during the execution of the task to be processed; taking the file sending task "Help me send the AA file on my desktop to BB through the XX application" as an example, the first task execution parameter can include the file path information "desktop / AA file" and the file receiving object "BB". Taking the TV series reservation task "Help me reserve RRR Season 3 through the YY application" as an example, the first task execution parameter can include the reservation TV series name "RRR Season 3".

[0055] In one specific embodiment, the pre-defined intelligent agent can perform semantic understanding of the task instruction information by calling a large language model to obtain the task intent information corresponding to the task to be processed, the application identifier to be run corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed.

[0056] In an optional embodiment, the above-mentioned inputting task intent information, application identifier to be run, and first task execution parameters into a preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed may include: Input the task intent information, the application identifier to be run, and the first task execution parameters into the preset intelligent agent. Obtain the preset operation chain set corresponding to the target application based on the application identifier to be run. Determine the matching operation chain information corresponding to the task to be processed from the preset operation chain set based on the task intent information. Generate the task operation chain information based on the matching operation chain information and the first task execution parameters.

[0057] In a specific embodiment, a set of preset operation chains can be stored in advance based on the application identifiers of different applications; correspondingly, the set of preset operation chains corresponding to the target application can be obtained by combining the identifier of the application to be run; specifically, the set of preset operation chains corresponding to the target application can be the operation chain information for implementing each function in the target application, that is, the set of preset operation chains corresponding to the target application can include the operation required for each step, the control of the operation (such as buttons, input boxes, etc.), and the interface change path information after the operation, from the starting page of the target application to the completion of a specific function (such as the file sending function corresponding to the file sending task).

[0058] In one specific embodiment, the operation chain information (operation link information) corresponding to the task target (application function) indicated by the task intent information in the preset operation chain set can be used as the matching operation chain information corresponding to the task to be processed. That is, the matching operation chain information can be the operation link information in the process of executing the task to be processed based on the target application.

[0059] In a specific embodiment, generating task operation chain information based on the matching operation chain information and the first task execution parameters may include: binding the first task execution parameters to the corresponding task operation in the matching operation chain information to obtain the task operation chain information. That is, the task operation chain information can be the operation chain information after binding the corresponding task execution parameters to the matching operation chain information.

[0060] In the above embodiments, the task intent information, the identifier of the application to be run, and the first task execution parameters are input into a preset intelligent agent. First, a preset operation chain set corresponding to the target application is obtained based on the identifier of the application to be run. Then, based on the task intent information, the matching operation chain information corresponding to the task to be processed is determined from the preset operation chain set. Next, based on the matching operation chain information and the first task execution parameters, task operation chain information is generated. This can convert the user's task requirements into operation chain information during the execution of the task by the target application, realize intelligent parsing of operation chain information, and thus effectively cope with different task requirements and improve the diversity of cross-terminal task processing scenarios.

[0061] In practical applications, there are situations where certain task execution parameters cannot be parsed from the task instruction information. Optionally, during the semantic understanding of the task instruction information, the preset intelligent agent can, in addition to performing intent recognition, application identification, and task parameter parsing, also identify the additional execution parameters required to realize the task intent information. Accordingly, the above-mentioned semantic understanding of the task instruction information by the preset intelligent agent to obtain the task intent information corresponding to the task to be processed, the application identifier to be run corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed may include: Based on the semantic understanding of the task instruction information by the preset intelligent agent, the task intent information, the identifier of the application to be run, the first task execution parameters and the parameter supplementation instruction information are obtained. In one specific embodiment, the aforementioned parameter supplementation information can be used to indicate the supplementary task execution parameters required to execute the task to be processed. In one specific embodiment, taking the flight booking task "Help me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon through the ZZ application" as an example, the pre-defined intelligent agent, during the semantic understanding of the task execution information, can determine the first task execution parameters, including the travel time "XX month XX day, around 15:00 in the afternoon," the origin "NN," and the destination "MM." Since the flight booking task also requires the user to select a payment method and enter a payment password, correspondingly, parameter supplementation information can be determined to indicate the supplementary payment method and payment password required to execute the flight booking task.

[0062] In the above embodiments, during the semantic understanding of task instruction information, the preset intelligent agent can identify the execution parameters to be supplemented in order to realize the task intent information, in addition to performing intent recognition, application recognition and task parameter parsing. This can effectively ensure the success rate of subsequent task execution and the reliability of cross-terminal collaboration, and can also better cope with different task requirements and improve the diversity of cross-terminal task processing scenarios.

[0063] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the method may further include: Display supplementary parameter prompts in the dialogue interface between the current account and the preset agent; In response to the parameter supplementation command, obtain the execution parameters for the second task; Accordingly, the task intent information, the application identifier to be run, and the first task execution parameters are input into a preset intelligent agent to decompose the task operation, and the resulting task operation chain information corresponding to the task to be processed may include: The task intent information, the application identifier to be run, the first task execution parameters, and the second task execution parameters are input into a preset intelligent agent to decompose the task operation and obtain the task operation chain information.

[0064] In a specific embodiment, taking the above-mentioned task of "helping me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon" through the ZZ application as an example, as follows: Figure 6 As shown, Figure 6This is a schematic diagram of a dialog interface displaying parameter supplement prompts provided in an embodiment of this application; specifically, the information corresponding to 601 can be parameter supplement prompts; optionally, the user can supplement the payment method through the corresponding payment method selection control; and input the payment password by entering the password in the corresponding input box.

[0065] In one specific embodiment, the parameter supplementation instruction can be an instruction for supplementing the task execution parameters indicated by the parameter supplementation prompt information; specifically, the second task execution parameters can be the task execution parameters indicated by the parameter supplementation prompt information. Optionally, the second task execution parameters can be supplemented through dialog messages, or supplementation controls (such as selection controls, input boxes, etc.) can be displayed during the parameter supplementation prompt information display process; correspondingly, the user can combine the supplementation controls to supplement the second task execution parameters.

[0066] In a specific embodiment, the task intent information, the application identifier to be run, the first task execution parameters, and the second task execution parameters are input into a preset intelligent agent to decompose the task operation and obtain the detailed task operation chain information. For a more detailed explanation of this process, please refer to the above description of inputting the task intent information, the application identifier to be run, and the first task execution parameters into a preset intelligent agent to decompose the task operation and obtain the detailed task operation chain information corresponding to the task to be processed. This will not be repeated here. That is, when binding task execution parameters, the bound task execution parameters include both the first and second task execution parameters.

[0067] In the above embodiments, displaying parameter supplement prompts in the dialogue interface between the current account and the preset intelligent agent can facilitate users to supplement task execution parameters, thereby ensuring the effectiveness and accuracy of the task operation chain information obtained from the task operation decomposition, improving the reliability of task execution, and better responding to different task requirements, thus enhancing the diversity of cross-terminal task processing scenarios.

[0068] In a specific embodiment, taking the file sending task described above, "Help me send the AA file on my desktop to BB via the XX application," as an example, the task operation chain information may include OpenApp (XX application); Click (Contacts tab); InputText (Search box "BB"); Click (BB in the search results); Click ("+" sign in the chat window); Click (File options); SelectFile (Desktop / AA file); Click (Send button).

[0069] In a specific embodiment, taking the above-mentioned task of "help me reserve RRR Season 3 through YY application" as an example, the task operation chain information may include OpenApp (YY application); InputText (search box, "RRR Season 3"); Click (search control); Click (reservation control corresponding to RRR Season 3 in the search results).

[0070] In a specific embodiment, taking the above-mentioned task of "helping me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon through the ZZ app" as an example, the supplementary second task execution parameters include payment method 1 and payment password XXXXXX; the task operation chain information may include OpenApp (ZZ app); InputText (starting location input box, "NN"); InputText (destination input box, "MM"); InputText (date input box, "XX month XX day"); Click (search control); Click (option corresponding to around 15:00 in the flight search results); Click (selection control corresponding to payment method 1); InputText (input box, "XXXXXX").

[0071] In one specific embodiment, when the second terminal corresponding to the target application is a mobile terminal, i.e., the target application is an application running in the operating system environment corresponding to the mobile terminal, the task operation chain information can be converted into any command executable by an application in the operating system environment corresponding to the mobile terminal, such as UIAutomator (User Interface Automator) command or XCUITest (Xcode UI Test) command; when the second terminal corresponding to the target application is a PC terminal, i.e., the target application is an application running in the operating system environment corresponding to the PC terminal, the task operation chain information can be converted into any command executable by an application in the operating system environment corresponding to the PC terminal, such as Win32 API (Windows 32-bit Application Programming Interface) message command or UIAutomation (User Interface Automation) command.

[0072] In the above embodiments, the task instruction information is first semantically understood based on a preset intelligent agent to obtain the task intent information corresponding to the task to be processed, the application identifier to be run (the application identifier of the target application) corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed. Then, the task intent information, the application identifier to be run, and the first task execution parameters are input into the preset intelligent agent for task operation decomposition to obtain the task operation chain information corresponding to the task to be processed. Based on the preset intelligent agent, the task operation chain information is mapped to obtain the task instruction sequence. Through semantic understanding and action decomposition mechanism, the user's task instructions can be automatically parsed into specific operation sequences, realizing the automatic parsing of the user's task instruction information into specific task instruction sequences. The user only needs to issue an intent and wait for the result, without having to manually operate the cross-terminal device, which can greatly improve the operational convenience and efficiency of cross-terminal task processing. Moreover, it can be combined with intelligent parsing capabilities to enhance the diversity of cross-terminal task processing scenarios.

[0073] S205: Deploy the target application in a local simulator instance to obtain a simulator instance that hosts the target application.

[0074] In one specific embodiment, the local emulator instance can be an emulator instance on the current execution subject; for example, if the first terminal is the execution subject, the local emulator instance can be an emulator instance on the first terminal; if the server is the execution subject, the local emulator instance can be an emulator instance on the server; specifically, deploying the target application on the local emulator instance can include downloading and logging into the target application. Specifically, the login account and password corresponding to the target application can be pre-set; optionally, the user can also set the login account and password corresponding to the target application during task execution, for example, after the user enters task instructions.

[0075] In an optional embodiment, the above method may further include: Idle simulator instances are determined from a preset resource pool based on a preset intelligent agent; Accordingly, the above-mentioned deployment of the target application in a local emulator instance, resulting in an emulator instance hosting the target application, can include: Deploy the target application in a local idle emulator instance to obtain an emulator instance that hosts the target application.

[0076] In one specific embodiment, at least one simulator instance can be pre-created on the first terminal, and the corresponding status information can be registered with a preset intelligent agent (i.e., the simulator scheduling and mapping unit in the task agent engine). The preset resource pool may include at least one simulator instance created on the first terminal. Specifically, the status information corresponding to any simulator instance may include information that can identify the simulator instance, such as the simulator instance's IP (Internet Protocol) address, port, and system version.

[0077] In a specific embodiment, in the initial state, all simulator instances are idle simulator instances (i.e., idle simulator instances). The preset intelligent agent can record whether each simulator instance is idle based on the corresponding state information. When any simulator instance is assigned to a task and the corresponding application is deployed, the idle status information of the simulator instance is updated from idle to non-idle, so that the idle simulator instances can be accurately determined in the subsequent simulator instance allocation process.

[0078] In the above embodiments, while combining intelligent parsing of task instruction information with intelligent agents, the intelligent agents allocate idle simulator instances from the preset resource pool to the target application corresponding to the current task to be processed. This can optimize resource utilization, improve task reliability, and better improve task execution efficiency.

[0079] S207: Execute a sequence of task instructions based on a simulator instance that carries the target application.

[0080] In one specific embodiment, a sequence of task instructions can be sent to the simulator instance using an instruction transmission tool. For example, the instruction transmission tool can be configured according to actual application requirements, such as any instruction transmission tool like ADB (Android Debug Bridge) or Xcode (Xcode development tool), to send the sequence of task instructions to the simulator instance.

[0081] In one specific embodiment, the simulator instance is equipped with an automated executor, which can be used to control the target application to automatically execute the task instruction sequence corresponding to the task to be processed; accordingly, the above-mentioned execution of the task instruction sequence based on the simulator instance carrying the target application may include: The automated executor controls the target application in the simulator instance to automatically execute a sequence of task instructions through the simulator instance's system interface.

[0082] In one specific embodiment, the system interface of the simulator instance is a control channel provided by the operating system running on the simulator instance, enabling interface operations, element manipulation, and application behavior control for the target application. Specifically, the system interface of the simulator instance may include a UI (User Interface) automation service interface, an input event interface, and an application component management interface; the UI automation service interface can be used to obtain interface element information to achieve control recognition and interface manipulation. The input event interface can be used to inject simulated input events such as touch and key presses. The application component management interface can be used to control the application's startup, navigation, and running status.

[0083] In the above embodiments, the automated actuator realizes automated control of internal applications through the system interface in the simulator instance, which can improve the efficiency and security of cross-platform task processing scenarios.

[0084] In an optional embodiment, the above-described sequence of task instructions based on a simulator instance carrying the target application may include: Based on the simulator instance hosting the target application, execute the task instructions in the task instruction sequence sequentially, and perform the following steps during the execution of any task operation instruction: Get the current interface control tree of the current application interface corresponding to any task operation instruction. The current interface control tree is used to indicate the layout of controls in the current application interface. Based on the current interface control tree, locate the operation control corresponding to any task operation instruction; Execute any task operation command based on the operation controls.

[0085] In one specific embodiment, based on a simulator instance carrying a target application, executing task instructions in a sequence of task instructions sequentially may include: an automated executor controlling the target application to execute task operation instructions in a sequence of task instructions sequentially through the system interface of the simulator instance.

[0086] In a specific embodiment, the current interface control tree can be a tree structure with controls in the current application interface as nodes and hierarchical relationships between controls as edges. This current interface control tree is used to indicate the layout of controls in the current application interface, specifically, the layout of controls such as buttons and input boxes.

[0087] In one specific embodiment, the current UI control tree can be obtained from the application server corresponding to the target application; alternatively, the current UI control tree can also be obtained in real time during task execution. Specifically, the automated executor can directly read the current UI data from the system running within the simulator instance at runtime, thereby generating the latest UI control tree (current UI control tree), thus ensuring that even if the application version is updated or the interface is redesigned, the system can still obtain the latest UI control tree.

[0088] In a specific embodiment, the operation control corresponding to any task operation instruction can be the control required for the operation during the execution of the task operation instruction. Specifically, when the operation control corresponding to any task operation instruction is determined, the task operation corresponding to the task operation instruction can be executed in conjunction with the operation control. For example, taking the above-mentioned task of "help me reserve RRR Season 3 through YY application" as an example, if the current application interface is the search results interface for RRR Season 3, the corresponding task operation instruction can be the instruction to click the reservation control corresponding to RRR Season 3, that is, the corresponding operation control is to click the reservation control corresponding to RRR Season 3. Accordingly, after locating the reservation control corresponding to RRR Season 3, the operation of "clicking the reservation control corresponding to RRR Season 3" can be executed.

[0089] In the above embodiments, based on the simulator instance carrying the target application, during the process of sequentially executing the task operation instructions in the task instruction sequence, the current interface control tree of the current application interface corresponding to any task operation instruction can be obtained, and the operation control corresponding to the task operation instruction can be located based on the current interface control tree. This can effectively ensure the accuracy of operation control location, and the task operation instruction can be executed based on the operation control, thereby improving the accuracy of task execution control and thus improving task execution efficiency.

[0090] In practical applications, there may be situations where the corresponding operation control cannot be located through the interface control tree. For example, a function button might be drawn on an image instead of a standard control, leading to control location failure. Feedback can be provided to the user so they can troubleshoot the issue promptly. Optionally, the task instruction information can be the dialogue information of the current account with a preset intelligent agent. The above method may also include: If the operation control corresponding to the target task instruction fails to be located, obtain the current interface image corresponding to the target task instruction and the task execution log information corresponding to the task to be processed. The current interface image and task execution log information are displayed in the dialogue interface between the current account and the preset intelligent agent.

[0091] In a specific embodiment, the target task instruction can be any corresponding operation control in the task instruction sequence that failed to locate the task operation instruction; the current interface image corresponding to the target task instruction can be the interface image of the current application interface during the execution of the target task instruction; optionally, the current interface image corresponding to the target task instruction can be obtained by means of screenshot or other methods.

[0092] In one specific embodiment, the task execution log information can record the task execution sequence, operation steps, running status, and execution results. Specifically, if the positioning of the operation control corresponding to the target task instruction fails, the task execution log information can include the execution result information of the failure to position the operation control corresponding to the target task instruction. Correspondingly, when the current interface image and task execution log information are displayed on the dialogue interface between the current account and the preset intelligent agent, the user can intuitively understand the interface corresponding to the task execution failure and the reason for the failure by combining the current interface image.

[0093] In the above embodiments, when the positioning of the operation control corresponding to the target task instruction fails, the current interface image corresponding to the target task instruction and the task execution log information corresponding to the task to be processed are obtained, and the current interface image and task execution log information are displayed on the dialogue interface between the current account and the preset intelligent agent. This makes it easier for users to intuitively understand the situation of task execution interruption caused by the failure to position a specific operation control, and greatly improves the convenience of users to troubleshoot anomalies.

[0094] In an optional embodiment, if the corresponding operation control cannot be located through the interface control tree, the operation control can be relocated through control identification. Accordingly, the above method may further include: If the location of the operation control corresponding to the target task instruction fails, obtain the current interface image corresponding to the target task instruction; Perform control recognition on the current interface image to determine the location information of the operation control corresponding to the target task instruction in the current interface image; The operation controls corresponding to the target task instructions are located based on the location information; The target task instruction is executed based on the operation control corresponding to the target task instruction.

[0095] In a specific embodiment, the target task instruction can be any corresponding operation control in the task instruction sequence that failed to locate the task operation instruction; the current interface image corresponding to the target task instruction can be the interface image of the current application interface during the execution of the target task instruction; optionally, the current interface image corresponding to the target task instruction can be obtained by means of screenshot or other methods.

[0096] In one specific embodiment, a preset control recognition model can be used to recognize controls in the current interface image to determine the location information of the operation control corresponding to the target task instruction in the current interface image; specifically, the preset control recognition model can be a pre-trained artificial intelligence model for control recognition.

[0097] In a specific embodiment, the control description information of the operation control corresponding to the target task instruction can be obtained; specifically, the control description information can be information describing the operation control corresponding to the target task instruction. For example, the control description information can include at least one of the following: control shape description information or control content description information; optionally, the control description information of the operation control corresponding to the target task instruction can be input into a preset control recognition model for control recognition to obtain the positioning information of the operation control corresponding to the target task instruction in the current interface image.

[0098] In one specific embodiment, if the location information of the operation control corresponding to the target task instruction is found in the current interface image, the location information can be passed to the automated executor so that the automated executor can locate the operation control corresponding to the target task instruction in the current application interface corresponding to the target task instruction based on the location information; and execute the target task instruction based on the operation control corresponding to the target task instruction.

[0099] In the above embodiments, when the location of the operation control corresponding to the target task instruction fails, the current interface image corresponding to the target task instruction is obtained, and control recognition is performed on the current interface image to determine the location information of the operation control corresponding to the target task instruction in the current interface image. Then, the operation control corresponding to the target task instruction can be located based on the location information, and the target task instruction can continue to be executed. By performing control recognition on the interface image and relocating the operation control, automatic relocation of the control can be achieved in the case of location failure, which greatly improves the reliability of task execution and can also improve task execution efficiency.

[0100] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the above method may further include: During the execution of the task instruction sequence, the task execution progress information corresponding to the task to be processed is displayed in the dialogue interface between the current account and the preset intelligent agent; In one specific embodiment, the task execution progress information can be information used to indicate the execution progress of the task to be processed. Specifically, the task execution progress information includes at least one of the following: execution progress description text during the execution of the task instruction sequence, or execution interface image during the execution of the task instruction sequence.

[0101] In one specific embodiment, the execution progress description text can be the description text of the task operation corresponding to the executed task operation instruction during the execution of the task instruction sequence. The execution interface image can be the interface image of the application interface corresponding to the executed task operation instruction during the execution of the task instruction sequence.

[0102] In a specific embodiment, taking the file sending task described above, "Help me send the AA file on my desktop to BB via the XX application," as an example, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of a dialog interface provided in this application embodiment to display task execution progress information corresponding to tasks to be processed. Figure 1 Specifically, the information corresponding to 701 can be the task execution progress information of the file sending task, and the task execution progress information includes the execution progress description text of the file sending task, and the current execution is up to the "click on contacts" operation.

[0103] In a specific embodiment, taking the aforementioned task of "helping me reserve RRR Season 3 through the YY application" as an example, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of a dialog interface provided in this application embodiment to display task execution progress information corresponding to tasks to be processed. Figure 2 Specifically, the information corresponding to 801 can be the task execution progress information of the TV series reservation task, and the task execution progress information includes the execution progress description text of the TV series reservation task and the corresponding execution interface image, and the current execution is up to the operation of "entering RRR Season 3 in the search box".

[0104] In a specific embodiment, taking the above-mentioned task of "helping me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon" through the ZZ application as an example, as follows: Figure 9 As shown, Figure 9 This is a schematic diagram of a dialog interface provided in this application embodiment to display task execution progress information corresponding to tasks to be processed. Figure 3 Specifically, the information corresponding to 901 can be the task execution progress information of the flight booking task, and the task execution progress information includes the execution progress description text of the flight booking task, and the current execution is up to the operation of "enter NN in the starting address input box".

[0105] In the above embodiments, during the execution of the task instruction sequence, the task execution progress information corresponding to the task to be processed is displayed on the dialogue interface between the current account and the preset intelligent agent. The task execution progress information includes at least one of the following: execution progress description text during the execution of the task instruction sequence, or execution interface image during the execution of the task instruction sequence. This can improve the diversity and flexibility of the execution progress presentation format, and also allow users to understand the task execution progress without real-time cumbersome operations, allowing users to transform from operators to supervisors, and truly realize the efficiency improvement of cross-terminal collaboration.

[0106] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the above method may further include: If the pending task is successfully executed, the task success indication information corresponding to the pending task will be displayed in the dialogue interface between the current account and the preset intelligent agent. In one specific embodiment, the task success indication information can be information indicating that the task to be processed has been successfully completed. Specifically, the task success indication information includes task success prompt text, or at least one of the task completion interface images corresponding to the task to be processed in the target application.

[0107] In one specific embodiment, the task success notification text can be text indicating that the task to be processed has been successfully executed. For example, it can include at least one of task success description text or successful operation description text. The successful operation description text can be a description text of the task operation corresponding to the successfully executed task operation instruction. The task completion interface image can be an interface image of the application interface corresponding to the completion of the task to be processed. For example, taking the above-mentioned file sending task "Help me send the AA file on my desktop to BB through the XX application", where the XX application is a social communication application (i.e., sending the AA file through the dialogue interface between the current account and BB), the task completion interface image can be the dialogue interface between the current account and BB, and the dialogue interface includes the dialogue information of the current account sending the AA file to BB.

[0108] In a specific embodiment, taking the file sending task described above, "Help me send the AA file on my desktop to BB via the XX application," as an example, such as... Figure 10 As shown, Figure 10 This is a schematic diagram of a dialog interface provided in this application embodiment, displaying a success indication message for a pending task. Figure 1 Specifically, the information corresponding to 1001 can be the task success indication information for the file sending task, and the task success indication information includes the task success prompt text for the file sending task and the task completion interface image for the file sending task.

[0109] In a specific embodiment, taking the aforementioned task of "helping me reserve RRR Season 3 through the YY application" as an example, such as... Figure 11 As shown, Figure 11 This is a schematic diagram of a dialog interface provided in this application embodiment, displaying a success indication message for a pending task. Figure 2 Specifically, the information corresponding to 1101 can be the task success indication information for the TV series reservation task, and the task success indication information includes the task success prompt text for the TV series reservation task and the task completion interface image for the TV series reservation task.

[0110] In a specific embodiment, taking the above-mentioned task of "helping me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon" through the ZZ application as an example, as follows: Figure 12 As shown, Figure 12 This is a schematic diagram of a dialog interface provided in this application embodiment, displaying a success indication message for a pending task. Figure 3 Specifically, the information corresponding to 1201 can be the task success indication information for the flight booking task, and the task success indication information includes the task success prompt text for the flight booking task.

[0111] In the above embodiments, when the task to be processed is successfully executed, the task success indication information corresponding to the task to be processed is displayed on the dialogue interface between the current account and the preset intelligent agent. The task success indication information includes at least one of the following: task success prompt text, or the task end interface image corresponding to the task to be processed in the target application. This can improve the diversity and flexibility of the execution progress presentation, and also allow users to understand the task execution status without too much interaction, so that users can turn from operators to supervisors, and truly realize the efficiency improvement of cross-terminal collaboration.

[0112] In an optional embodiment, completed tasks can be recorded to facilitate further processing by the user or to view and manage historical tasks.

[0113] In a specific embodiment, taking the file sending task described above, "Help me send the AA file on my desktop to BB via the XX application," and taking the first terminal as the execution subject as an example, such as... Figure 13 As shown, Figure 13 This is a schematic diagram of a cross-platform task processing procedure according to an exemplary embodiment. Figure 1Specifically, when the current account inputs the task instruction information "Help me send the AA file on my desktop to BB through the XX application", the task instruction information "Help me send the AA file on my desktop to BB through the XX application" can be input into a preset intelligent agent. The preset intelligent agent performs semantic understanding on the task instruction information to obtain the task intent information "send file", the identifier of the application to be run "the identifier of the XX application", and the first task execution parameters "file path information "desktop / AA file" and file receiving object "BB"" corresponding to the file sending task. Then, the preset intelligent agent decomposes the task operation based on the task intent information "send file", the identifier of the application to be run "the description of the XX application", and the first task execution parameters "file path information "desktop / AA file" and file receiving object "BB"" to obtain the task operation chain information "OpenApp (XX application), Click (contacts tab), InputText (search box "BB"), Click (BB in the search results), Click ("+" sign in the chat window), Click (file option), SelectFile (desktop / AA file), Click (send button)". Next, the pre-defined agent maps the task operation chain information "OpenApp (XX application), Click (contacts tab), InputText (search box "BB"), Click (BB in search results), Click ("+" sign in chat window), Click (file options), SelectFile (desktop / AA file), Click (send button)" to obtain the task instruction sequence corresponding to the file sending task. Then, it sends the task instruction sequence corresponding to the file sending task to the emulator instance that hosts the XX application, and executes the task instruction sequence based on the emulator instance that hosts the XX application to complete the file sending task.

[0114] In a specific embodiment, taking the aforementioned task of "helping me reserve RRR Season 3 through the YY application" as an example, and using the server as the execution entity, such as... Figure 14 As shown, Figure 14 This is a schematic diagram of a cross-platform task processing procedure according to an exemplary embodiment. Figure 2Specifically, if the current account can input the task instruction information "Help me send the AA file on the desktop to BB through the XX application" on the first terminal, the first terminal can send the task instruction information "Help me send the AA file on the desktop to BB through the XX application" to the server. The server can input the task instruction information "Help me reserve RRR Season 3 through the YY application" into the preset intelligent agent. The preset intelligent agent performs semantic understanding on the task instruction information to obtain the task intent information "Reserve TV series" corresponding to the TV series reservation task, the identifier of the application to be run corresponding to the TV series reservation task "Identifier of YY application", and the first task execution parameter "Reserved TV series name "RRR Season 3"" corresponding to the TV series reservation task. Then, the preset intelligent agent on the server decomposes the task operation based on the task intent information "Reserve TV series", the identifier of the application to be run "Identifier of YY application", and the first task execution parameter "Reserved TV series name "RRR Season 3"" to obtain the task operation chain information "OpenApp (YY application), InputText (search box, "RRR Season 3"), Click (search control), Click (reservation control corresponding to RRR Season 3 in the search results)". Next, the pre-defined agent on the server maps the task operation chain information "OpenApp (YY application), InputText (search box, "RRR Season 3"), Click (search control), Click (reservation control corresponding to RRR Season 3 in the search results)" to obtain the task instruction sequence corresponding to the TV series reservation task. Then, it sends the task instruction sequence corresponding to the TV series reservation task to the simulator instance on the server that hosts the YY application, and executes the task instruction sequence based on the simulator instance that hosts the YY application to complete the TV series reservation task.

[0115] In a specific embodiment, taking the aforementioned task of "booking a flight from NN to MM on XX month XX day, around 15:00 in the afternoon via the ZZ application" as an example, and assuming that the first terminal and the server jointly execute the task, as follows: Figure 15 As shown, Figure 15 This is a schematic diagram of a cross-platform task processing procedure according to an exemplary embodiment. Figure 3Specifically, if the current account can input the task instruction information "Help me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon through the ZZ application" on the first terminal, the first terminal can send the task instruction information "Help me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon through the ZZ application" to the server. The server can input the task instruction information "Help me book a flight from NN to MM on XX month XX day, around 15:00 in the afternoon through the ZZ application" into a preset intelligent agent. The preset intelligent agent performs semantic understanding on the task instruction information and obtains the task intent information "book a flight", the identifier of the application to be run "the identifier of the ZZ application" corresponding to the flight booking task, and the first task execution parameter "travel time "XX month XX day, around 15:00 in the afternoon" corresponding to the flight booking task. The system first obtains the parameter supplementation instructions for the origin "NN" and destination "MM" and the corresponding flight booking task. Then, it displays the parameter supplementation prompt in the dialogue interface between the current account and the preset intelligent agent. Upon triggering the parameter supplementation command by the current account, it obtains the second task execution parameters "Payment Method 1" and "Payment Password" corresponding to the flight booking task. Further, the preset intelligent agent decomposes the task operation based on the task intent information "Flight Booking," the application identifier "ZZ Application Identifier," the first task execution parameter "Travel Time XX Month XX Day, around 15:00 PM," the origin "NN" and destination "MM," and the second task execution parameters "Payment Method 1" and "Payment Password," to obtain the task operation chain information "OpenApp (ZZ Application)" corresponding to the flight booking task. InputText(starting location input box, "NN location"); InputText(destination input box, "MM location"); InputText(date input box, "XX month XX day"); Click(search control); Click(options for around 15:00 in flight search results); Click(selection control for payment method 1); InputText(input box, "XXXXXX").Next, the pre-defined intelligent agent maps the task operation chain information "OpenApp (ZZ application); InputText (starting location input box, "NN location"); InputText (destination input box, "MM location"), InputText (date input box, "XX month XX day"), Click (search control), Click (option corresponding to around 15:00 in flight search results); Click (selection control corresponding to payment method 1), InputText (input box, "XXXXXX")" to obtain the task instruction sequence corresponding to the flight booking task. This task instruction sequence is then sent to the first terminal. The first terminal then executes the task instruction sequence based on the simulator instance that hosts the ZZ application locally to complete the flight booking task.

[0116] Furthermore, it should be noted that the multiple or various types mentioned in the embodiments of this application can be at least two or at least two kinds.

[0117] As can be seen from the technical solutions provided in the embodiments of this specification above, in the process of executing tasks corresponding to the target application on the second terminal, this specification can achieve task parsing based on the preset intelligent agent by inputting task instruction information on the first terminal in the form of dialogue information input to a preset intelligent agent, thereby obtaining operation instructions (task instruction sequence) for the execution of the task to be processed based on the target application. This realizes the automatic parsing of the user's task instruction information into a specific operation sequence. After deploying the target application in a local simulator instance and obtaining a simulator instance carrying the target application, the task instruction sequence is executed based on the simulator instance carrying the target application. This allows the application task execution on the second terminal side to be triggered on the first terminal side without the user manually operating the application on the second terminal side, without switching devices, greatly improving the efficiency and convenience of task processing. Furthermore, by connecting the intelligent task processing capabilities on the first terminal side with the application execution capabilities on the second terminal side, it can effectively cope with different cross-terminal task processing needs, greatly reduce the operation and time costs of cross-terminal task processing, and improve the efficiency and interactive convenience of cross-terminal collaborative processing.

[0118] This application also provides a cross-platform task processing device, such as... Figure 16 As shown, Figure 16 This is a schematic diagram of a cross-platform task processing device provided in an embodiment of this application. The device includes: The task instruction information acquisition module 1610 is configured to acquire the task instruction information corresponding to the task to be processed. The task instruction information is the dialogue information entered by the current account on the first terminal for a preset intelligent agent. The task parsing module 1620 is configured to perform task parsing on task indication information based on a preset intelligent agent to obtain a task instruction sequence. The task instruction sequence is at least one task operation instruction in the process of executing the task to be processed based on the target application, and the target application is the application corresponding to the second terminal. Application deployment module 1630 is configured to deploy the target application in a local simulator instance, thereby obtaining a simulator instance that hosts the target application; The task execution module 1640 is configured to execute a sequence of task instructions based on a simulator instance that hosts the target application.

[0119] In an optional embodiment, the task parsing module 1620 includes: The semantic understanding unit is configured to perform semantic understanding of the task instruction information based on a preset intelligent agent, and obtain the task intent information corresponding to the task to be processed, the application identifier to be run corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed. The application identifier to be run is the application identifier of the target application. The task decomposition unit is configured to input the task intent information, the application identifier to be run, and the first task execution parameters into a preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed. The instruction mapping unit is configured to perform instruction mapping based on a preset intelligent agent to the task operation chain information, thereby obtaining a task instruction sequence.

[0120] In an optional embodiment, the task decomposition unit is specifically configured to input task intent information, application identifier to be run, and first task execution parameters into a preset intelligent agent, obtain a preset operation chain set corresponding to the target application based on the application identifier to be run, determine the matching operation chain information corresponding to the task to be processed from the preset operation chain set based on the task intent information, and generate task operation chain information based on the matching operation chain information and the first task execution parameters.

[0121] In an optional embodiment, the semantic understanding unit is further configured to perform semantic understanding of the task instruction information based on a preset intelligent agent to obtain task intent information, application identifier to be run, first task execution parameters and parameter supplementation instruction information. The parameter supplementation indication information is used to indicate the additional task execution parameters required to execute the task to be processed.

[0122] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the above-mentioned device further includes: The first information display module is configured to display supplementary parameter prompts in the dialogue interface between the current account and the preset intelligent agent. The parameter supplementation module is configured to respond to the parameter supplementation command and obtain the execution parameters for the second task. The task decomposition unit is also configured to decompose the task operation by inputting the task intent information, the application identifier to be run, the first task execution parameters and the second task execution parameters into a preset intelligent agent, thereby obtaining the task operation chain information.

[0123] In an optional embodiment, the above-described apparatus further includes: The simulator instance determination module is configured to determine an idle simulator instance from a preset resource pool based on a preset agent; The application deployment module 1630 is also configured to deploy the target application in a local idle simulator instance, resulting in a simulator instance that hosts the target application.

[0124] In an optional embodiment, the task execution module 1640 is specifically configured to execute task instructions in a sequence of task instructions based on a simulator instance carrying the target application, and to perform the following steps during the execution of any task operation instruction: Get the current interface control tree of the current application interface corresponding to any task operation instruction. The current interface control tree is used to indicate the layout of controls in the current application interface. Based on the current interface control tree, locate the operation control corresponding to any task operation instruction; Execute any task operation command based on the operation controls.

[0125] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the above-mentioned device further includes: The information acquisition module is configured to acquire the current interface image corresponding to the target task instruction and the task execution log information corresponding to the task to be processed when the operation control corresponding to the target task instruction fails to be located. The target task instruction is any task operation instruction in the task instruction sequence whose corresponding operation control failed to be located. The second information display module is configured to display the current interface image and task execution log information on the dialogue interface between the current account and the preset intelligent agent.

[0126] In an optional embodiment, the above-described apparatus further includes: The interface image acquisition module is configured to acquire the current interface image corresponding to the target task instruction when the operation control corresponding to the target task instruction fails to be located. The target task instruction is any task operation instruction in the task instruction sequence for which the corresponding operation control fails to be located. The location information determination module is configured to perform control recognition on the current interface image and determine the location information of the operation control corresponding to the target task instruction in the current interface image. The control positioning module is configured to execute the operation control corresponding to the task instruction to locate the target based on the positioning information. The instruction execution module is configured to execute the target task instruction based on the operation control corresponding to the target task instruction.

[0127] In an optional embodiment, the simulator instance is provided with an automation executor, and the task execution module 1640 is further configured to execute the automation executor to control the target application in the simulator instance to automatically execute a sequence of task instructions through the system interface of the simulator instance.

[0128] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the above-mentioned device further includes: The execution progress display module is configured to display the task execution progress information corresponding to the task to be processed in the dialogue interface between the current account and the preset intelligent agent during the execution of the task instruction sequence. The task execution progress information includes at least one of the following: execution progress description text during the execution of the task instruction sequence, or execution interface images during the execution of the task instruction sequence.

[0129] In an optional embodiment, the task instruction information is the dialogue information of the current account to a preset intelligent agent, and the above-mentioned device further includes: The success indication information display module is configured to display the task success indication information corresponding to the pending task in the dialogue interface between the current account and the preset intelligent agent when the pending task is successfully executed. The task success indication information includes at least one of the following: task success message text, or the task completion interface image corresponding to the task to be processed in the target application.

[0130] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0131] Figure 17 This is a block diagram of an electronic device for cross-platform task processing provided in an embodiment of this application. The description takes the electronic device as a terminal as an example, and its internal structure diagram is as follows: Figure 17As shown. This electronic device may include an RF (Radio Frequency) circuit 1710, a memory 1720 including one or more computer-readable storage media, an input unit 1730, a display unit 1740, a sensor 1750, an audio circuit 1760, a WiFi (Wireless Fidelity) module 1770, a processor 1780 including one or more processing cores, and a power supply 1790, among other components. Those skilled in the art will understand that... Figure 17 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: RF circuit 1710 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 1780 for processing; additionally, it transmits uplink data to the base station. Typically, RF circuit 1710 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. Furthermore, RF circuit 1710 can also communicate wirelessly with networks and other terminals. Wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0132] The memory 1720 can be used to store software programs and modules. The processor 1780 executes various functional applications and data processing by running the software programs and modules stored in the memory 1720. The memory 1720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for the functions, etc.; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 1720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1720 may also include a memory controller to provide access to the memory 1720 for the processor 1780 and the input unit 1730.

[0133] Input unit 1730 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, input unit 1730 may include touch-sensitive surface 1731 and other input devices 1732. Touch-sensitive surface 1731, also known as a touch display screen or touchpad, can collect user touch operations on or near it (such as user operations using fingers, styluses, or any suitable object or accessory on or near touch-sensitive surface 1731), and drive corresponding connection devices according to a pre-set program. Optionally, touch-sensitive surface 1731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 1780, and can receive and execute commands from processor 1780. In addition, the touch-sensitive surface 1731 can be implemented using various methods such as resistive, capacitive, infrared, and surface acoustic wave. Besides the touch-sensitive surface 1731, the input unit 1730 may also include other input devices 1732. Specifically, other input devices 1732 may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick. The display unit 1740 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the terminal. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit 1740 may include a display panel 1741, which may optionally be configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or similar display panel. Further, a touch-sensitive surface 1731 may cover the display panel 1741. When the touch-sensitive surface 1731 detects a touch operation on or near it, it transmits the information to the processor 1780 to determine the type of touch event. Subsequently, the processor 1780 provides corresponding visual output on the display panel 1741 according to the type of touch event. The touch-sensitive surface 1731 and the display panel 1741 can be two independent components to implement input and output functions; however, in some embodiments, the touch-sensitive surface 1731 and the display panel 1741 can be integrated to achieve both input and output functions.

[0134] The terminal may also include at least one sensor 1750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1741 according to the ambient light level, and the proximity sensor can turn off the display panel 1741 and / or the backlight when the terminal is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that identify the terminal's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured on the terminal, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0135] Audio circuitry 1760, speaker 1761, and microphone 1762 provide an audio interface between the user and the terminal. Audio circuitry 1760 converts received audio data into electrical signals, which are then transmitted to speaker 1761, where they are converted into sound signals for output. Conversely, microphone 1762 converts collected sound signals into electrical signals, which are received by audio circuitry 1760, converted back into audio data, and then processed by processor 1780 before being transmitted via RF circuitry 1710 to, for example, another terminal, or output to memory 1720 for further processing. Audio circuitry 1760 may also include an earphone jack to facilitate communication between a peripheral headset and the terminal.

[0136] WiFi is a short-range wireless transmission technology. This terminal, through the WiFi module 1770, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 17 WiFi module 1770 is shown, but it is understood that it is not an essential component of the terminal and can be omitted as needed without changing the nature of the invention.

[0137] The processor 1780 is the control center of the terminal, connecting various parts of the terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 1720, and calls data stored in the memory 1720 to perform various functions and process data, thereby enabling overall monitoring of the terminal. Optionally, the processor 1780 may include one or more processing cores; preferably, the processor 1780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1780.

[0138] The terminal also includes a power supply 1790 (such as a battery) to power various components. Preferably, the power supply can be logically connected to the processor 1780 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1790 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0139] Although not shown, the terminal may also include a camera, Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the terminal is a touch screen display, and the terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors of the instructions in the method embodiment of the present invention.

[0140] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the cross-device task processing method as described in the embodiments of this disclosure.

[0141] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the cross-device task processing method of the present disclosure embodiments. In an exemplary embodiment, a computer program product or computer program is also provided, the computer program product including instructions stored in a computer-readable storage medium, the processor retrieving the instructions from the computer-readable storage medium and executing the instructions to implement the cross-end task processing method provided in the various optional implementations described above.

[0142] It is understood that in the specific implementation of this application, user-related data is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0144] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0145] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A cross-platform task processing method, characterized in that, The method includes: Obtain task instruction information corresponding to the task to be processed. The task instruction information is the dialogue information entered by the current account on the first terminal for a preset intelligent agent. Based on the preset intelligent agent, the task instruction information is parsed to obtain a task instruction sequence. The task instruction sequence is at least one task operation instruction in the process of the target application executing the task to be processed. The target application is the application corresponding to the second terminal. Deploy the target application in a local emulator instance to obtain an emulator instance that hosts the target application; The task instruction sequence is executed based on the simulator instance that carries the target application.

2. The method according to claim 1, characterized in that, The step of parsing the task indication information based on the preset intelligent agent to obtain the task instruction sequence includes: Based on the semantic understanding of the task indication information by the preset intelligent agent, the task intent information corresponding to the task to be processed, the application identifier to be run corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed are obtained, wherein the application identifier to be run is the application identifier of the target application. The task intent information, the application identifier to be run, and the first task execution parameters are input into the preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed. Based on the preset intelligent agent, the task operation chain information is mapped to obtain the task instruction sequence.

3. The method according to claim 2, characterized in that, The step of inputting the task intent information, the application identifier to be run, and the first task execution parameters into the preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed includes: The task intent information, the application identifier to be run, and the first task execution parameters are input into the preset intelligent agent. Based on the application identifier to be run, a preset operation chain set corresponding to the target application is obtained. Based on the task intent information, the matching operation chain information corresponding to the task to be processed is determined from the preset operation chain set. Based on the matching operation chain information and the first task execution parameters, task operation chain information is generated.

4. The method according to claim 2, characterized in that, The step of performing semantic understanding on the task indication information based on the preset intelligent agent to obtain the task intent information corresponding to the task to be processed, the application identifier to be run corresponding to the task to be processed, and the first task execution parameters corresponding to the task to be processed includes: Based on the preset intelligent agent, the task instruction information is semantically understood to obtain the task intent information, the application identifier to be run, the first task execution parameters and parameter supplementation instruction information; The parameter supplementation indication information is used to indicate the supplementary task execution parameters required to execute the task to be processed.

5. The method according to claim 4, characterized in that, The method further includes: The parameter supplement prompt information is displayed in the dialogue interface between the current account and the preset intelligent agent; In response to the parameter supplementation command, obtain the execution parameters for the second task; The step of inputting the task intent information, the application identifier to be run, and the first task execution parameters into the preset intelligent agent to decompose the task operation and obtain the task operation chain information corresponding to the task to be processed includes: The task intent information, the application identifier to be run, the first task execution parameters, and the second task execution parameters are input into the preset intelligent agent to decompose the task operation and obtain the task operation chain information.

6. The method according to claim 2, characterized in that, The method further includes: Based on the preset intelligent agent, an idle simulator instance is determined from the preset resource pool; Deploying the target application in a local emulator instance to obtain an emulator instance hosting the target application includes: Deploy the target application in the local idle simulator instance to obtain a simulator instance that hosts the target application.

7. The method according to any one of claims 1 to 6, characterized in that, The execution of the task instruction sequence based on the simulator instance carrying the target application includes: Based on the simulator instance carrying the target application, the task instructions in the task instruction sequence are executed sequentially, and the following steps are performed during the execution of any task operation instruction: Obtain the current interface control tree of the current application interface corresponding to any task operation instruction; the current interface control tree is used to indicate the layout of controls in the current application interface. Based on the current interface control tree, locate the operation control corresponding to any task operation instruction; Execute any of the task operation instructions based on the operation controls.

8. The method according to claim 7, characterized in that, The method further includes: If the operation control corresponding to the target task instruction fails to be located, the current interface image corresponding to the target task instruction and the task execution log information corresponding to the task to be processed are obtained. The target task instruction is any task operation instruction in the task instruction sequence for which the operation control corresponding to the task instruction fails to be located. The current interface image and the task execution log information are displayed in the dialogue interface between the current account and the preset intelligent agent.

9. The method according to claim 7, characterized in that, The method further includes: If the operation control corresponding to the target task instruction fails to be located, the current interface image corresponding to the target task instruction is obtained, wherein the target task instruction is any task operation instruction in the task instruction sequence for which the operation control corresponding to the target task instruction fails to be located. Perform control recognition on the current interface image to determine the location information of the operation control corresponding to the target task instruction in the current interface image; Based on the location information, locate the operation control corresponding to the target task instruction; The target task instruction is executed based on the operation control corresponding to the target task instruction.

10. The method according to any one of claims 1 to 6, characterized in that, The simulator instance is equipped with an automated executor, and the execution of the task instruction sequence based on the simulator instance hosting the target application includes: The automated executor controls the target application in the simulator instance to automatically execute the task instruction sequence through the system interface of the simulator instance.

11. The method according to any one of claims 1 to 6, characterized in that, The method further includes: During the execution of the task instruction sequence, the task execution progress information corresponding to the task to be processed is displayed in the dialogue interface between the current account and the preset intelligent agent. The task execution progress information includes at least one of the following: execution progress description text during the execution of the task instruction sequence, or execution interface images during the execution of the task instruction sequence.

12. The method according to any one of claims 1 to 6, characterized in that, The method further includes: If the pending task is successfully executed, the task success indication information corresponding to the pending task will be displayed in the dialogue interface between the current account and the preset intelligent agent. The task success indication information includes at least one of the following: task success message text, or the task completion interface image corresponding to the task to be processed in the target application.

13. A cross-platform task processing device, characterized in that, The device includes: The task instruction information acquisition module is configured to acquire the task instruction information corresponding to the task to be processed. The task instruction information is the dialogue information of the current account inputting on the first terminal for a preset intelligent agent. The task parsing module is configured to perform task parsing on the task indication information based on the preset intelligent agent to obtain a task instruction sequence. The task instruction sequence is at least one task operation instruction in the process of executing the task to be processed based on the target application. The target application is the application corresponding to the second terminal. The application deployment module is configured to deploy the target application in a local simulator instance to obtain a simulator instance that hosts the target application; The task execution module is configured to execute the task instruction sequence based on the simulator instance carrying the target application.

14. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the cross-platform task processing method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the cross-device task processing method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes instructions stored in a computer-readable storage medium, from which a processor retrieves and executes the instructions to implement the cross-platform task processing method as described in any one of claims 1 to 12.

Images