Task processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a task processing method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving a to-be-executed instruction sequence sent by a target deep learning model; for each to-be-executed instruction, complementing the dynamic variable of the current to-be-executed instruction according to the global context information of the current to-be-executed instruction to obtain an executable instruction, and sending the executable instruction to a target execution agent for execution; and receiving an execution result which is fed back by the target execution agent and corresponds to the current instruction to be executed, and feeding back the execution result to the target deep learning model in real time, so that after all the instructions to be executed are executed, the target deep learning model sends the summarized execution result to a user display interface. When the RPA instruction is processed, context hosting is carried out by obtaining global context information, and intermediate result data is effectively stored, so that the effect of ensuring the accuracy of data in an RPA operation process is achieved.

Description

Technical Field

[0001] This invention relates to the field of process automation technology, and in particular to a task processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, Robotic Process Automation (RPA) technology has developed rapidly and become widespread, and has been widely used in various industries.

[0003] Currently, when applying RPA technology, users can automate processes through RPA commands controlled by natural language, or generate RPA process code based on natural language deep learning models according to natural language commands. However, due to the different RPA use cases of various companies and the characteristics of different computer operating system versions, the underlying code and API interfaces of different RPA tools vary greatly, resulting in poor universality of RPA technology. Furthermore, the need to manually pass context data between RPA process steps can lead to inaccurate data during the use of RPA technology.

[0004] To address the aforementioned issues, it is necessary to improve the RPA-based task processing methods. Summary of the Invention

[0005] This invention provides a task processing method, apparatus, electronic device, and storage medium to address the poor usability and versatility of existing RPA technology, as well as the potential for inaccurate data during the RPA process.

[0006] In a first aspect, embodiments of the present invention provide a task processing method, including:

[0007] Receive a sequence of instructions to be executed sent by a target deep learning model; wherein the sequence of instructions to be executed corresponds to an execution task generated based on a natural language command input by a user, and the sequence of instructions to be executed includes at least one instruction to be executed, and each instruction to be executed corresponds to an execution subtask;

[0008] For each instruction to be executed, the dynamic variables of the current instruction to be executed are completed according to the global context information of the current instruction to be executed to obtain an executable instruction, and the executable instruction is sent to the target execution agent for execution; wherein, the global context information includes at least one of the following: the execution results of all instructions to be executed before the current instruction to be executed, real-time parameters of the system environment, user-defined configuration files, and inter-instruction association data;

[0009] The system receives the execution result corresponding to the currently pending instruction from the target execution agent and feeds the execution result back to the target deep learning model in real time. After all pending instructions have been executed, the target deep learning model sends the summarized execution result to the user display interface.

[0010] Secondly, embodiments of the present invention also provide a task processing apparatus, comprising:

[0011] A receiving module is used to receive a sequence of instructions to be executed sent by the target deep learning model; wherein the sequence of instructions to be executed includes at least one instruction to be executed, and each instruction to be executed corresponds to an execution subtask;

[0012] The instruction sending module is used to complete the dynamic variables of each instruction to be executed based on the global context information of the current instruction to be executed, so as to obtain an executable instruction, and send the executable instruction to the target execution agent for execution; wherein, the global context information includes at least one of the following: the execution results of all instructions to be executed before the current instruction to be executed, real-time system environment parameters, user-defined configuration files, and inter-instruction association data;

[0013] The execution result feedback module is used to receive the execution result corresponding to the current instruction to be executed from the target execution agent, and to feed the execution result back to the target deep learning model in real time, so that after all the instructions to be executed are completed, the target deep learning model will send the summarized execution result to the user display interface.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method described in any embodiment of the present invention.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the task processing method described in any embodiment of the present invention.

[0019] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the task processing method as described in any of the embodiments of the present invention.

[0020] The technical solution of this invention involves receiving a sequence of instructions to be executed sent by a target deep learning model; for each instruction to be executed, completing the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to obtain an executable instruction, and sending the executable instruction to the target execution agent for execution; receiving the execution result corresponding to the current instruction to be executed from the target execution agent, storing it in the global context information of the instruction, and feeding back a summary of the execution result to the target deep learning model in real time, so that after all the instructions to be executed are completed, the target deep learning model sends the summarized execution result to the user display interface. In this technical solution, after receiving the sequence of instructions to be executed sent by the target deep learning model, the sequence of instructions to be executed is sent to the RPA virtual machine for execution. Specifically, for each instruction to be executed, an executable instruction can be generated by obtaining the global context information corresponding to the instruction to be executed, and the executable instruction is sent to the execution agent connected to the RPA server, so that the execution agent feeds back the corresponding execution result after executing the executable instruction, which makes up for the defect of the target deep learning model's inability to remember complex data and ensures the security and reliability of the context data. This invention addresses the issue of context management in RPA processes caused by the lack of consideration for global and intermediate variables in existing technologies. It achieves context management by acquiring global context information when processing RPA instructions, effectively saving intermediate result data, and thus ensuring the accuracy of data in the RPA operation process.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a task processing method provided in Embodiment 1 of the present invention;

[0024] Figure 2This is a flowchart illustrating a specific example of a task processing method provided in Embodiment 1 of the present invention;

[0025] Figure 3 This is a schematic diagram of the RPA virtual machine architecture provided in Embodiment 1 of the present invention;

[0026] Figure 4 This is a flowchart of a task processing method provided according to Embodiment 2 of the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of a task processing device according to Embodiment 3 of the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the task processing method of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention. The acquisition, transmission, storage, use, and processing of data in the technical solutions of this application comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, certain software, components, or models and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solutions of this application, but it does not mean that the applicant has or necessarily used such solutions.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0031] Example 1

[0032] Figure 1This is a flowchart of a task processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to training an initial deep learning model based on an RPA instruction set to obtain a target deep learning model. After a user inputs a natural language command, the target deep learning model generates a corresponding task to be executed and generates a sequence of instructions to be executed corresponding to the task. Further, the sequence of instructions to be executed is sent to an RPA server, where the RPA virtual machine in the RPA server sequentially converts each instruction to be executed into executable instructions. The executable instructions are then sent to the execution agent connected to the RPA server for processing. After the execution agent completes its execution, the execution result is fed back to the RPA server, which then stores the execution result in the RPA virtual machine for hosting and feeds back a result summary to the target deep learning model. The target deep learning model summarizes all the execution results and then displays the visualized execution results to the user interface of the target user. This method can be executed by a task processing device, which can be implemented in hardware and / or software and can be configured in a computing device that executes the task processing method.

[0033] like Figure 1 As shown, the method includes:

[0034] S110, Receive the sequence of instructions to be executed sent by the target deep learning model.

[0035] In this technical solution, the sequence of instructions to be executed refers to the RPA instruction sequence, which corresponds to the task to be executed generated based on the natural language command input by the user. The sequence of instructions to be executed includes at least one instruction to be executed, and each instruction to be executed corresponds to a subtask to be executed.

[0036] The target deep learning model can be a natural language deep learning model trained on a pre-collected RPA instruction set. This target deep learning model receives natural language commands from the user, generates tasks to be executed based on the commands, and generates corresponding instruction sequences to be executed based on the tasks. These instruction sequences are then sent to the RPA virtual machine in the RPA server for processing. The RPA instructions translated into JSON format are parsed into JSON data with a unified format by the virtual machine in the RPA server, which adds variables according to the process context.

[0037] The task to be executed includes at least one subtask to be executed, and the sequence of instructions to be executed includes instructions to be executed corresponding to each subtask to be executed.

[0038] In practical applications, such as Figure 2As shown, when using RPA technology, users can input natural language commands through the user interface. After the natural language commands are transmitted to the large language model (i.e., the target deep learning model), the target deep learning model will perform flow planning based on the natural language commands to generate tasks to be executed. Further, the target deep learning model decomposes the tasks to be executed into at least one subtask to be executed, and generates RPA instructions (i.e., instructions to be executed) corresponding to each subtask. The instructions to be executed are then ordered according to the execution order of each subtask to obtain the sequence of instructions to be executed. After obtaining the sequence of instructions to be executed, it is sent to the RPA virtual machine, so that the instructions in the sequence are executed sequentially by the RPA virtual machine.

[0039] Based on this, the RPA virtual machine receives the sequence of instructions to be executed sent by the target deep learning model, and executes the sequence of instructions to be executed based on the RPA virtual machine.

[0040] S120. For each instruction to be executed, complete the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to be executed to obtain the executable instruction, and send the executable instruction to the target execution agent for execution.

[0041] The global context information includes at least one of the following: the execution results of all instructions to be executed before the currently executed instruction, real-time system environment parameters, user-defined configuration files, and inter-instruction association data.

[0042] In practical applications, please refer to [link / reference]. Figure 2 The RPA server is responsible for receiving the sequence of instructions to be executed sent by the target deep learning model and forwarding it to the RPA virtual machine. In order to realize the automated processing of the RPA process, when executing each instruction, it is necessary to obtain the global context information corresponding to the instruction and complete the dynamic variables required by the instruction based on the global context information, assemble the executable instruction, and send the executable instruction to the RPA agent (i.e., the execution agent) that is connected to the RPA server. The execution agent calls the platform interface, converts the executable instruction into a specific RPA operation, executes it, obtains the execution result, and sends the execution result to the RPA virtual machine in the RPA server for storage and hosting (i.e., writing the process context into the RPA virtual machine).

[0043] In a specific example, at the start of the task processing flow, the RPA virtual machine allocates a server-side `server_context` to store global parameters, variables, and attributes (i.e., global context information) within the task processing flow. Based on this, when executing RPA instructions (i.e., executing the instructions to be executed), the RPA virtual machine transmits each instruction sequentially from the RPA virtual machine to the execution agent for execution, and determines whether the execution was successful based on the return result.

[0044] RPA instructions include conditional statements, branching structures, and looping structures. When processing flow branches, the RPA server sends Boolean conditional statements to the execution agents and returns the results to the RPA server for branch selection. Furthermore, the RPA server maintains the network connection for at least one execution agent and sends executable instructions to the target agent, so that the target agent can return the execution results to the RPA server after execution. The RPA server then saves the received execution results in the `server_context`, packages them as a `prompt`, and sends it back to the target deep learning model.

[0045] Optionally, the dynamic variables of the instruction to be executed are completed based on the global context information of the instruction to be executed to obtain an executable instruction, and the executable instruction is sent to the target execution agent for execution. This includes: if the instruction to be executed is complete, obtaining the global context information corresponding to the instruction to be executed and sending the instruction to be executed as an executable instruction to the target execution agent; if the instruction to be executed is incomplete, completing the dynamic variables of the instruction to be executed based on the global context information corresponding to the instruction to be executed to obtain an executable instruction, and sending the instruction to be executed as an executable instruction to the target execution agent.

[0046] Executable instructions refer to those generated by the RPA server after completing the dynamic variables of the instruction to be executed based on global context information. These executable instructions are sent by the RPA virtual machine to the target execution agent, which then executes them.

[0047] Specifically, when the processing flow of a task to be executed begins, the RPA server creates a context (server_context) to store the connection object (conn) between the control agent and the server, as well as various process variables (i.e., global context information). The RPA server can pass commands to the conn object, send messages to the execution agent, listen for data uploaded by the conn object, obtain the execution results fed back by the execution agent, and store the execution results on the RPA server as the global context information for the next instruction to be executed. This is how communication between the RPA server and the execution agent is achieved.

[0048] Understandably, RPA instructions involving conditional, branching, and looping structures require boolean variables for branch selection. These variables, retrieved from the execution agent, are stored in `server_context` for use during program jumps. During debugging, breakpoints can be inserted into the flow; the server parses these breakpoints and records the intermediate results returned at each breakpoint.

[0049] Optionally, after sending the executable instructions to the target execution agent for execution, the method further includes: executing the executable instructions based on the target execution agent, recording the data to be fed back corresponding to the executable instructions, encapsulating the data to be fed back to obtain the execution result, and feeding back the execution result to the RPA virtual machine.

[0050] The execution agent in the RPA virtual machine is an agent program installed on the target system platform. It can communicate with the RPA server and receive RPA instructions (i.e., executable instructions) sent by the RPA virtual machine in the RPA server one by one. Based on this, it transforms the executable instructions into specific operations and feeds back the instruction execution results to the RPA server.

[0051] In practical applications, such as Figure 3 As shown, in this RPA server architecture, data flow between the RPA server and execution agents is achieved through bidirectional communication. The RPA server issues executable instructions to each execution agent (e.g., the execution agent can be a Windows platform, Linux platform, or other platforms). These executable instructions carry information such as process definitions and configuration parameters, enabling the execution agents to clearly identify the automated operation to be performed upon receiving the instructions. After executing the executable instructions, the execution agents obtain feedback data, such as running progress, execution status, operation logs, and collected system data. They then encapsulate this feedback data to obtain the execution result and send it back to the server, forming a closed-loop management system. The overall data flow is centered on the RPA server, enabling unified scheduling and monitoring of the execution process across multiple execution agent nodes.

[0052] In a specific example, the execution agent performs logical analysis based on the received abstract executable instructions and implements the operations of the executable instructions by calling the local platform's API interface. Based on this, the execution agent communicates with the RPA server and returns the execution results to the RPA server, including whether the instruction was executed successfully and any error messages. Simultaneously, it establishes the execution agent's context (agent_context) to store the data objects, global parameters, variables, and attributes (i.e., global context information) obtained during previous instruction executions.

[0053] In other words, the execution agent proactively establishes a TCP connection with the RPA server. After the connection is established, it waits for the RPA server to send RPA instructions (i.e., executable instructions). Upon receiving the executable instructions, it calls the corresponding local methods for parsing and execution, obtaining the specific operation corresponding to the executable instructions, and then executes it. After the execution agent completes the specific operation corresponding to the executable instructions, it receives the feedback data, including the execution status, return value, and error information. The returned result is encapsulated into a JSON format and sent back to the RPA server. For example, the execution result may contain fields such as resStat, retVal, and errMsg. resStat represents the instruction execution status, with two types: SUCC and ERROR. retVal represents the return result of the instruction, generally a boolean type. errMsg represents the error information during instruction execution, which is a string.

[0054] S130: Receive the execution result corresponding to the currently pending instruction from the target execution agent, and feed the execution result back to the target deep learning model in real time, so that after all pending instructions have been executed, the target deep learning model will send the summarized execution result to the user display interface.

[0055] Based on the above example, after the RPA server obtains the return result from the execution agent, it encapsulates it into a prompt that the target deep learning model can recognize (i.e., an execution result that the target deep learning model can recognize), so that the target deep learning model can continue to execute subsequent instruction generation operations based on the received execution result, either sequentially or by jumping through the process according to the judgment conditions.

[0056] Based on the above example, after receiving the execution result corresponding to the currently pending instruction from the target execution agent, the method further includes: updating the global context information of the next pending instruction adjacent to the currently pending instruction based on the execution result, and using the next pending instruction as the current pending instruction; repeatedly executing the steps of completing the dynamic variables of the currently pending instruction according to the global context information of the currently pending instruction to obtain the executable instruction, sending the executable instruction to the target execution agent for execution, receiving the execution result corresponding to the currently pending instruction from the target execution agent, and feeding back the execution result to the target deep learning model in real time, until all pending instructions have been executed, and the target deep learning model sends the summarized execution result to the user display interface.

[0057] It's important to note that when the Agent executes executable instructions, it may need to utilize global variables or pass parameters, such as globally shared browser objects, or pass read table data to the next process (i.e., the processing of the next instruction to be executed). To address this, a context `agent_context` is created locally for each process within the Agent. `agent_context` is similar to the heap and stack in the JVM, storing global variables and passing relevant parameters. The result of each executable instruction is stored in the `ans` variable within `agent_context`. If the result needs to be passed to another variable, an assignment instruction is used to retrieve the value stored in `ans`. The purpose of `agent_context` is to store the data objects generated during the actual execution of the RPA process locally, uploading only references to the data objects to the RPA server instead of the data objects themselves. This reduces communication overhead, ensures the privacy of local data, and improves data security.

[0058] Based on this, continue to see Figure 2 The system processes the instructions to be executed in the sequence of instructions in turn until all instructions have been executed. Then, it feeds back the execution results of all instructions to the target deep learning model in turn, so that the target deep learning model can visualize the results of all instructions to be executed and display them to the user interface.

[0059] The technical solution of this invention involves receiving a sequence of instructions to be executed sent by a target deep learning model; for each instruction to be executed, completing the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to obtain an executable instruction, and sending the executable instruction to the target execution agent for execution; receiving the execution result corresponding to the current instruction to be executed from the target execution agent, and feeding the execution result back to the target deep learning model in real time, so that after all instructions to be executed are completed, the target deep learning model sends the summarized execution result to the user display interface. In this technical solution, after receiving the sequence of instructions to be executed sent by the target deep learning model, the sequence of instructions to be executed is sent to the RPA virtual machine for execution. Specifically, for each instruction to be executed, an executable instruction can be generated by obtaining the global context information corresponding to the instruction to be executed, and the executable instruction is sent to the execution agent connected to the RPA server, so that the execution agent feeds back the corresponding execution result after executing the executable instruction, thus compensating for the defect of the target deep learning model in being unable to remember complex data and ensuring the security and reliability of the context data. This invention addresses the issue of context management in RPA processes caused by the lack of consideration for global and intermediate variables in existing technologies. It achieves context management by acquiring global context information when processing RPA instructions, effectively saving intermediate result data, and thus ensuring the accuracy of data in the RPA operation process.

[0060] Example 2

[0061] Figure 4 The flowchart of a task processing method provided in Embodiment 2 of the present invention includes, optionally, the following steps before receiving the sequence of instructions to be executed sent by the target deep learning model: obtaining at least one set of instructions to be trained corresponding to a task processing domain; wherein the set of instructions to be trained includes at least one of variable definition instruction group, variable assignment instruction group, interface processing operation instruction group, human-computer interaction operation instruction group, and client operation instruction group; and inputting the set of instructions to be trained into an initial deep learning model for training to obtain a target deep learning model corresponding to each task processing domain.

[0062] like Figure 4 As shown, the method includes:

[0063] S210. Obtain at least one set of instructions to be trained corresponding to a task processing domain.

[0064] The training instruction set includes at least one of the following: variable definition instruction set, variable assignment instruction set, interface processing operation instruction set, human-computer interaction operation instruction set, and client operation instruction set.

[0065] In practical applications, RPA instructions are logical abstractions of basic RPA operations, separated from specific operating system platforms. To obtain a target deep learning model, before training the initial deep learning model, at least one RPA instruction set (i.e., the instruction set to be trained) corresponding to the task processing domain is acquired. This RPA instruction set is characterized by completeness and standardization.

[0066] Specifically, based on the division of task processing domains, RPA commands can be categorized into variable definition, browser operations, file operations, mouse and keyboard operations, and client operations. RPA commands include command ID, command name, command parameters, and command return value.

[0067] S220. Input the instruction set to be trained into the initial deep learning model for training to obtain the target deep learning model corresponding to each task processing domain.

[0068] The initial deep learning model is an untrained natural language deep learning model. The target deep learning model is the model obtained after training the initial deep learning model based on the instruction set to be trained.

[0069] Taking the training of an initial deep learning model using a set of instructions corresponding to one of the task processing domains as an example, the RPA instruction set (i.e., the training instruction set) corresponding to the task processing domain is obtained. The document containing the RPA instruction set is then input into the initial deep learning model to perform fine-tuning training, or alignment operations. The advantage of this approach is that, compared to directly training and fine-tuning the initial deep learning model based on the RPA process code generated by the RPA process, training the initial deep learning model based on the RPA instruction set significantly reduces the workload of training the model.

[0070] Specifically, during the fine-tuning of the initial deep learning model, hierarchical reinforcement learning was used to train the model, fine-tuning the model at two levels: task decomposition and instruction conversion within the RPA process. Hierarchical reinforcement learning is a branch of reinforcement learning that decomposes complex problems into several sub-problems, achieving the goal of solving the complex problem by solving each sub-problem one by one.

[0071] Optionally, before receiving the sequence of instructions to be executed sent by the target deep learning model, the method further includes: receiving natural language commands input by the target user and generating tasks to be executed based on the natural language commands; decomposing the tasks to be executed to obtain at least one subtask to be executed, and performing instruction conversion on each subtask to obtain the instructions to be executed corresponding to each subtask; generating a sequence of instructions to be executed based on all the instructions to be executed, and sending the sequence of instructions to be executed to the RPA virtual machine in the RPA server.

[0072] The RPA server includes an RPA virtual machine and at least one execution agent that communicates with the RPA server.

[0073] In practical applications, Prompt Engineering is a technique used to optimize and improve the performance of natural language deep learning models (i.e., the initial deep learning model). During training, appropriate prompts can be designed to guide the model to better understand and process user input, thereby improving its performance and achieving the target deep learning model. Prompt Engineering is typically applied to tasks such as text generation, classification, and reasoning. In short, a prompt for a natural language deep learning model refers to the initial text input provided by the user or system to the initial deep learning model during interaction, guiding the model to generate corresponding responses or perform specific tasks (i.e., generating tasks to be executed based on natural language commands input by the target user).

[0074] Specifically, the initial deep learning model's instruction generation prompt is divided into two levels: a prompt for planning the RPA operation process and a prompt for generating interactive RPA operation instructions. Based on the target user's needs and natural language commands, the RPA operation process prompt (i.e., the task to be executed) is planned and marked as prompt-1: Please plan the specific RPA operation process based on the user's natural language command {}. If there are any questions, consult the user before proceeding.

[0075] Furthermore, according to the planned RPA operation flow, interactive RPA operation instruction prompts (i.e., instructions to be executed corresponding to the subtasks to be executed) are generated step by step, and these instructions are marked as prompt-2: Please convert the natural language RPA operation flow into RPA instructions based on the RPA instruction library, and send them to the RPA virtual machine for execution. Refer to the RPA instruction library for instruction format. Then wait for the RPA virtual machine to return the result.

[0076] Based on this, the execution result returned by the RPA virtual machine to the target deep learning model will also be packaged in the form of a prompt, labeled as prompt-3. Simultaneously, the RPA virtual machine will manage the context variable `context` to ensure that data in various formats (such as CSV, images, and compressed files) can be stored correctly. Here, prompt-3 indicates: the execution status of step {} is {}, the execution result is stored in the variable {}, the data type of this variable is {}, please proceed to the next step.

[0077] Based on this, prompt-2 is used to convert the operation steps into specific executable instructions, which are then executed by function tools. When the function tools execute, they send the executable instructions to the RPA virtual machine, obtain global context information to get the RPA instructions that can actually be executed (i.e., executable instructions), and then send the executable instructions to the execution agent that is connected to the RPA server.

[0078] It should be noted that the return value of the instruction to be executed sent by the target deep learning model to the RPA virtual machine is just a code. The actual data is managed by the RPA virtual machine. The target deep learning model only needs to remember the code and pass the return value code to the next instruction to be executed. This reduces the memory burden of the target deep learning model and avoids the unreliability of the generated parameter data.

[0079] Finally, the return data of the RPA virtual machine is explained. The return data will be packaged according to prompt-3. For example, when executing instruction 3, the RPA virtual machine will return prompt: The execution status of step 3 is successful, and the execution result is stored in the variable userNameInput. The data type of this variable is String. Please proceed to the next step.

[0080] For example, when a browser is opened, the target deep learning model generates instructions to be executed and sends them to the RPA virtual machine. The RPA virtual machine manages the values ​​of the variables generated in the middle and provides feedback on the existence of such a variable in the target deep learning model, including its name and dynamic variable information.

[0081] Optionally, before receiving the natural language command input by the target user and generating the task to be executed based on the natural language command, the method further includes: when a trigger prompt of the user interface control is detected, generating a structured natural language instruction based on the operation performed by the target user on the user interface control; sending the natural language instruction to the target deep learning model and receiving the visualization feedback from the target deep learning model to obtain the execution result.

[0082] S230: Receive the sequence of instructions to be executed sent by the target deep learning model.

[0083] S240. For each instruction to be executed, complete the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to be executed to obtain the executable instruction, and send the executable instruction to the target execution agent for execution.

[0084] S250: Receive the execution result corresponding to the currently pending instruction from the target execution agent, and feed the execution result back to the target deep learning model in real time, so that after all pending instructions have been executed, the target deep learning model will send the summarized execution result to the user display interface.

[0085] For example, the target user issues a natural language command as "Open your browser and log in to website A, the URL is xxx.com, the username is xxx, and the password is xxx." Based on this natural language command, a task to be executed is generated, the task to be executed is broken down into at least one sub-task to be executed, and the instructions to be executed corresponding to each sub-task to be executed are submitted to the target deep learning model.

[0086] The target deep learning model will then output the following:

[0087] 1. Open your browser and enter the website address xxx.com;

[0088] 2. Check if the page is open;

[0089] 3. Locate the username input box and enter the username xxx;

[0090] 4. Locate the password input box and enter your password;

[0091] 5. Click the login button.

[0092] Based on this, the target depth model visualizes the execution results and provides feedback to the user's user interface.

[0093] The technical solution of this invention further includes, before receiving the sequence of instructions to be executed sent by the target deep learning model, the following steps: obtaining at least one set of instructions to be trained corresponding to a task processing domain; wherein the set of instructions to be trained includes at least one of variable definition instruction group, variable assignment instruction group, interface processing operation instruction group, human-computer interaction operation instruction group, and client operation instruction group; inputting the set of instructions to be trained into an initial deep learning model for training to obtain a target deep learning model corresponding to each task processing domain. Before receiving the sequence of instructions to be executed sent by the target deep learning model, the method further includes: receiving natural language commands input by a target user and generating tasks to be executed based on the natural language commands; decomposing the tasks to be executed to obtain at least one sub-task to be executed, and performing instruction conversion on each sub-task to obtain instructions to be executed corresponding to each sub-task; generating a sequence of instructions to be executed based on all instructions to be executed, and sending the sequence of instructions to be executed to the RPA virtual machine. Furthermore, after processing the sequence of instructions to be executed, the RPA virtual machine generates the corresponding executable instructions and sends them to the execution agent. After the execution agent returns the execution results, it feeds these results back to the target deep learning model. This allows the target deep learning model to aggregate the execution results of all the instructions to be executed and then send all the results to the user interface for visualization. In this technical solution, the target deep learning model is obtained by fine-tuning the initial deep learning model using the RPA instruction set. Compared to directly using the original RPA code for fine-tuning, this training method significantly reduces the workload of fine-tuning while improving the reliability of the target deep learning model. Secondly, in this technical solution, after generating executable instructions, the RPA virtual machine hands them over to the execution agent connected to the RPA server. By controlling the execution agent to perform specific operations on the instructions, it is not dependent on a specific programming language; it only needs to meet the functional requirements defined by the RPA instructions, giving RPA technology strong versatility and applicability.

[0094] Example 3

[0095] Figure 5 This is a schematic diagram of a task processing device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes: a receiving module 310, an instruction sending module 320, and an execution result feedback module 330.

[0096] The receiving module 310 is used to receive a sequence of instructions to be executed sent by the target deep learning model; wherein the sequence of instructions to be executed includes at least one instruction to be executed, and each instruction to be executed corresponds to an execution subtask;

[0097] The instruction sending module 320 is used to complete the dynamic variables of each instruction to be executed based on the global context information of the current instruction to be executed, so as to obtain an executable instruction, and send the executable instruction to the target execution agent for execution; wherein, the global context information includes at least one of the following: the execution results of all instructions to be executed before the current instruction to be executed, real-time parameters of the system environment, user-defined configuration files, and inter-instruction association data;

[0098] The execution result feedback module 330 is used to receive the execution result corresponding to the currently pending instruction from the target execution agent, and to feed the execution result back to the target deep learning model in real time, so that after all pending instructions have been executed, the target deep learning model will send the summarized execution result to the user display interface.

[0099] The technical solution of this invention involves receiving a sequence of instructions to be executed sent by a target deep learning model; for each instruction to be executed, completing the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to obtain an executable instruction, and sending the executable instruction to the target execution agent for execution; receiving the execution result corresponding to the current instruction to be executed from the target execution agent, and feeding the execution result back to the target deep learning model in real time, so that after all instructions to be executed are completed, the target deep learning model sends the summarized execution result to the user display interface. In this technical solution, after receiving the sequence of instructions to be executed sent by the target deep learning model, the sequence of instructions to be executed is sent to the RPA virtual machine for execution. Specifically, for each instruction to be executed, an executable instruction can be generated by obtaining the global context information corresponding to the instruction to be executed, and the executable instruction is sent to the execution agent connected to the RPA server, so that the execution agent feeds back the corresponding execution result after executing the executable instruction, thus compensating for the defect of the target deep learning model in being unable to remember complex data and ensuring the security and reliability of the context data. This invention addresses the issue of context management in RPA processes caused by the lack of consideration for global and intermediate variables in existing technologies. It achieves context management by acquiring global context information when processing RPA instructions, effectively saving intermediate result data, and thus ensuring the accuracy of data in the RPA operation process.

[0100] Optionally, the task processing device further includes: an instruction set acquisition module, used to acquire at least one training instruction set corresponding to a task processing domain before receiving the sequence of instructions to be executed sent by the target deep learning model; wherein the training instruction set includes at least one of variable definition instruction set, variable assignment instruction set, interface processing operation instruction set, human-computer interaction operation instruction set, and client operation instruction set.

[0101] The model training module is used to input the instruction set to be trained into the initial deep learning model for training, so as to obtain the target deep learning model corresponding to each task processing domain.

[0102] Optionally, the task processing device further includes: a task generation module, used to receive natural language commands input by the target user before receiving the sequence of instructions to be executed sent by the target deep learning model, and to generate tasks to be executed based on the natural language commands;

[0103] The instruction conversion module is used to decompose the task to be executed into at least one subtask to be executed, and to convert the instructions of each subtask to be executed to obtain the instruction to be executed corresponding to each subtask.

[0104] The instruction sequence generation module is used to generate an instruction sequence to be executed based on all instructions to be executed, and send the instruction sequence to be executed to the RPA virtual machine; wherein, the RPA server includes an interface for deep learning models, an RPA virtual machine, and at least one execution agent that communicates with the RPA server.

[0105] Optionally, the task processing device further includes: a natural language instruction generation module, used to generate structured natural language instructions based on the operation performed by the target user on the user interface control when a trigger prompt of the user interface control is detected before receiving the natural language command input by the target user and generating the task to be executed based on the natural language command;

[0106] The natural language instruction sending module is used to send natural language instructions to the target deep learning model and receive the execution result from the visualization of the feedback from the target deep learning model.

[0107] Optionally, the instruction sending module includes: a first sending unit, used to obtain global context information corresponding to the current instruction to be executed if the current instruction to be executed is complete, and send the instruction to be executed as an executable instruction to the target execution agent;

[0108] The second sending unit is used to complete the dynamic variables of the current instruction to be executed based on the global context information corresponding to the current instruction to be executed if the current instruction to be executed is incomplete, so as to obtain an executable instruction, and send the instruction to be executed as an executable instruction to the target execution agent.

[0109] Optionally, the task processing device is further configured to, after sending the executable instructions to the target execution agent for execution, execute the executable instructions based on the target execution agent, record the data to be fed back corresponding to the executable instructions, encapsulate the data to be fed back, and obtain the execution result; and feed back the execution result to the RPA virtual machine.

[0110] Optionally, the task processing device further includes: an update module, which, after receiving the execution result corresponding to the currently executed instruction from the target execution agent, updates the global context information of the next instruction to be executed adjacent to the currently executed instruction based on the execution result, and uses the next instruction to be executed as the currently executed instruction;

[0111] The repetitive execution module is used to repeatedly execute the steps of completing the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to be executed, obtaining the executable instruction, sending the executable instruction to the target execution agent for execution, receiving the execution result corresponding to the current instruction to be executed from the target execution agent, and feeding the execution result back to the target deep learning model in real time, until all instructions to be executed are completed, and the target deep learning model sends the summarized execution result to the user display interface.

[0112] The task processing device provided in the embodiments of the present invention can execute the task processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0113] Example 4

[0114] Figure 6 A schematic diagram of the structure of an electronic device 10 according to an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0115] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0117] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as task processing methods.

[0118] In some embodiments, the task processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the task processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the task processing method by any other suitable means (e.g., by means of firmware).

[0119] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the task processing methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0123] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0124] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0125] Example 5

[0126] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the task processing method provided in any embodiment of this application.

[0127] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0129] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A task processing method, characterized in that, include: Receive a sequence of instructions to be executed sent by a target deep learning model; wherein the sequence of instructions to be executed corresponds to an execution task generated based on a natural language command input by a user, and the sequence of instructions to be executed includes at least one instruction to be executed, and each instruction to be executed corresponds to an execution subtask; For each instruction to be executed, the dynamic variables of the current instruction to be executed are completed according to the global context information of the current instruction to be executed to obtain an executable instruction, and the executable instruction is sent to the target execution agent for execution; wherein, the global context information includes at least one of the following: the execution results of all instructions to be executed before the current instruction to be executed, real-time parameters of the system environment, user-defined configuration files, and inter-instruction association data; The system receives the execution result corresponding to the currently pending instruction from the target execution agent and feeds the execution result back to the target deep learning model in real time. After all pending instructions have been executed, the target deep learning model sends the summarized execution result to the user display interface.

2. The method according to claim 1, characterized in that, Before receiving the sequence of instructions to be executed sent by the target deep learning model, the method further includes: Obtain at least one set of instructions to be trained corresponding to a task processing domain; wherein, the set of instructions to be trained includes at least one of the following: variable definition instruction group, variable assignment instruction group, interface processing operation instruction group, human-computer interaction operation instruction group, and client operation instruction group; The instruction set to be trained is input into the initial deep learning model for training to obtain the target deep learning model corresponding to each task processing domain.

3. The method according to claim 2, characterized in that, Before receiving the sequence of instructions to be executed sent by the target deep learning model, the method further includes: Receive natural language commands input by the target user, and generate tasks to be executed based on the natural language commands; The task to be executed is decomposed to obtain at least one subtask to be executed, and each subtask to be executed is converted into an instruction to obtain an instruction to be executed corresponding to each subtask. A sequence of instructions to be executed is generated based on all instructions to be executed, and the sequence of instructions to be executed is sent to the RPA virtual machine in the RPA server; wherein the RPA server includes the RPA virtual machine and at least one execution agent that is communicatively connected to the PPA server.

4. The method according to claim 3, characterized in that, Before receiving the natural language command input by the target user and generating the task to be executed based on the natural language command, the method further includes: When a trigger prompt is detected for a user interface control, a structured natural language instruction is generated based on the operation performed by the target user on the user interface control. The natural language instructions are sent to the target deep learning model, and the execution result is obtained by receiving the visualization feedback from the target deep learning model.

5. The method according to claim 1, characterized in that, The step of completing the dynamic variables of the instruction to be executed based on the global context information of the instruction to be executed to obtain an executable instruction, and sending the executable instruction to the target execution agent for execution, includes: If the current instruction to be executed is complete, then obtain the global context information corresponding to the current instruction to be executed, and send the instruction to be executed as an executable instruction to the target execution agent; If the current instruction to be executed is incomplete, the dynamic variables of the current instruction to be executed are completed based on the global context information corresponding to the current instruction to be executed to obtain an executable instruction, and the instruction to be executed is sent to the target execution agent as an executable instruction.

6. The method according to claim 1, characterized in that, After sending the executable instructions to the target execution agent for execution, the method further includes: The execution agent executes the executable instructions based on the target, records the data to be fed back corresponding to the executable instructions, and encapsulates the data to be fed back to obtain the execution result. The execution results are then fed back to the RPA server.

7. The method according to claim 1, characterized in that, After receiving the execution result corresponding to the currently pending instruction from the target execution agent, the method further includes: Based on the execution result, update the global context information of the next instruction to be executed adjacent to the current instruction to be executed, and use the next instruction to be executed as the current instruction to be executed; The process of repeatedly completing the dynamic variables of the current instruction to be executed based on the global context information of the current instruction to be executed, obtaining an executable instruction, sending the executable instruction to the target execution agent for execution, receiving the execution result corresponding to the current instruction to be executed from the target execution agent, and feeding the execution result back to the target deep learning model in real time, continues until all instructions to be executed are completed. The target deep learning model then sends the summarized execution result to the user display interface.

8. A task processing device, characterized in that, include: A receiving module is used to receive a sequence of instructions to be executed sent by the target deep learning model; wherein the sequence of instructions to be executed includes at least one instruction to be executed, and each instruction to be executed corresponds to an execution subtask; The instruction sending module is used to complete the dynamic variables of each instruction to be executed based on the global context information of the current instruction to be executed, so as to obtain an executable instruction, and send the executable instruction to the target execution agent for execution; wherein, the global context information includes at least one of the following: the execution results of all instructions to be executed before the current instruction to be executed, real-time system environment parameters, user-defined configuration files, and inter-instruction association data; The execution result feedback module is used to receive the execution result corresponding to the current instruction to be executed from the target execution agent, and to feed the execution result back to the target deep learning model in real time, so that after all the instructions to be executed are completed, the target deep learning model will send the summarized execution result to the user display interface.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the task processing method according to any one of claims 1-7.

Images