Frameworks and agent loops for automatically generating tools and / or building artificial intelligence agents that use those tools.

Abstract

The present invention provides a computing system, method, and non-temporary computer-readable medium that automatically generates tools that have effective interoperability with other applications and software functions, and provides a framework for building AI agents that use these tools. [Solution] The agent loop is expressed in Business Process Model and Notation (BPMN) or any other desired workflow language, and the tools are implemented as static, call-specific, or dynamic caller-side or server-side tools, and may not be defined at all, may be defined only abstractly, or may be fully defined within the AI ​​agent definition. An AI agent implementing these tools can also build other AI agents that can leverage the caller-side and server-side tools.

Description

[Technical Field]

[0001] The present invention relates to software automation in general, and more specifically to a framework and agent loop for building artificial intelligence (AI) agents for automatically generating tools and / or using those tools. [Background technology]

[0002] AI agents are employed that possess memory, knowledge bases (e.g., repositories of policies and context grounding, which are collections of data, documents, etc., that an organization provides for its specific operations and needs), generative AI and large-scale language model (LLM) capabilities enabling natural language communication, and decision-making capabilities based on historical information. For example, a user can send natural language queries to an AI agent via prompts, and the AI ​​agent can provide responses based on its knowledge base and LLM. These AI agents also have human-in-the-loop capabilities, allowing them to query human users about queries they cannot adequately address. However, these AI agents tend to be independently developed, proprietary software applications and do not have effective interoperability with other applications and software functions. Therefore, improved and / or alternative approaches may be beneficial. [Overview of the project]

[0003] Some embodiments of the present invention may provide solutions to technical challenges and needs that have not yet been adequately identified, recognized, or addressed by current software automation technologies, and / or may provide useful alternatives thereto. For example, some embodiments of the present invention relate to frameworks and agent loops for automatically generating tools and / or building AI agents that use those tools.

[0004] In one embodiment, one or more non-temporary computer-readable media are configured to store one or more computer programs and to cause at least one processor to initialize an AI agent configured to automatically generate a tool. The one or more computer programs are further configured to cause at least one processor to enter an agent loop for the AI ​​agent in response to the successful initialization of the AI ​​agent. The one or more computer programs are further configured to cause at least one processor to execute a tool automatically. The tool includes the definition and implementation of an application programming interface (API).

[0005] In another embodiment, the computer implementation method includes the computing system entering an agent loop for an AI agent. The computer implementation method further includes the computing system automatically executing a tool. The tool includes an API definition and implementation. The tool is a static, call-specific, or dynamic caller-side or server-side tool.

[0006] In yet another embodiment, one or more computing systems include memory for storing computer program instructions and at least one processor configured to execute the stored computer program instructions. The computer program instructions are configured to cause the at least one processor to act to initialize an artificial intelligence (AI) agent configured to automatically generate a tool. The computer program instructions are further configured to cause the at least one processor to act to enter an agent loop for the AI ​​agent in response to the successful initialization of the AI ​​agent. The computer program instructions are further configured to cause the at least one processor to act to automatically execute a tool. The tool includes an API definition and implementation comprising an executable file, another AI agent, or robotic process automation (RPA). The tool is a static, call-specific, or dynamic caller-side or server-side tool. [Brief explanation of the drawing]

[0007] To facilitate understanding of the advantages of some embodiments of the present invention, a more detailed description of the invention, as briefly described above, will be given with reference to specific embodiments shown in the accompanying drawings. It should be understood that these drawings only illustrate typical embodiments of the invention and do not limit its scope. The present invention will be described more specifically and in detail with reference to the accompanying drawings.

[0008] [Figure 1] Figure 1 is an architectural diagram showing a hyperautomation system configured to perform agent-based automation and orchestration according to one embodiment of the present invention.

[0009] [Figure 2]FIG. 2 is a diagram illustrating some integrated functions of an artificial intelligence (AI) agent and a robotic process automation (RPA) robot according to an embodiment of the present invention.

[0010] [Figure 3] FIG. 3 is a diagram illustrating an AI agent, an RPA robot, an agent-based orchestration process (AOP), and a pool of applications according to an embodiment of the present invention.

[0011] [Figure 4A] FIG. 4A is a diagram illustrating an example of an AI agent service interface according to an embodiment of the present invention. [Figure 4B] FIG. 4B is a diagram illustrating an example of an AI agent service interface according to an embodiment of the present invention.

[0012] [Figure 5] FIG. 5 is a diagram illustrating an example of an AOP development interface according to an embodiment of the present invention.

[0013] [Figure 6] FIG. 6 is a diagram illustrating an example of an RPA development interface according to an embodiment of the present invention.

[0014] [Figure 7] FIG. 7 is a diagram illustrating an end-to-end AI agent, RPA robot, and AOP development and deployment system according to an embodiment of the present invention.

[0015] [Figure 8] FIG. 8 is an architecture diagram illustrating an agent-based automation and RPA system according to an embodiment of the present invention.

[0016] [Figure 9] FIG. 9 is an architecture diagram illustrating a deployed RPA system according to an embodiment of the present invention.

[0017] [Figure 10] Figure 10 is an architectural diagram showing the relationship between the designer, activity, and driver in one embodiment of the present invention.

[0018] [Figure 11] Figure 11 is an architectural diagram showing a computing system configured to implement a framework and agent loop for building an AI agent that automatically generates tools and / or uses those tools, according to one embodiment of the present invention.

[0019] [Figure 12A] Figure 12A shows an example of a framework and an example of a neural network trained to complement the agent loop for building an AI agent that automatically generates tools and / or uses those tools, according to one embodiment of the present invention.

[0020] [Figure 12B] Figure 12B shows an example of a neuron according to one embodiment of the present invention.

[0021] [Figure 13] Figure 13 is an architecture diagram showing a reference architecture of a generative AI model according to one embodiment of the present invention.

[0022] [Figure 14] Figure 14 is a flowchart showing the process of training an AI / machine learning (ML) model according to one embodiment of the present invention.

[0023] [Figure 15] Figure 15 shows an agent-based workflow of an AI agent according to one embodiment of the present invention.

[0024] [Figure 16]Figure 16 is a flowchart illustrating the process of implementing a framework and agent loop for building an AI agent to automatically generate tools and / or use those tools, according to one embodiment of the present invention.

[0025] Unless otherwise indicated, the same reference numerals consistently indicate the corresponding features throughout the attached drawings. [Modes for carrying out the invention]

[0026] Some embodiments relate to frameworks and agent loops for automatically generating tools and / or building AI agents that use those tools. In some embodiments, the agent loop may be expressed in Business Process Model and Notation (BPMN) or any other desired workflow language. In this specification, “Tool” means the definition and implementation of an Application Programming Interface (API) (e.g., written in a programming language and compiled into an executable form, another AI agent expressed in natural language, Robotic Process Automation (RPA), etc.).

[0027] Tools can be implemented as static, call-specific, or dynamic caller-side or server-side tools. Static server-side tools are fully, specifically, and explicitly defined in the AI ​​agent definition and cannot be modified for a particular AI agent call. Call-specific server-side tools are provided with an abstract definition in the AI ​​agent definition, but their API is not described. In some embodiments, the API definition may be provided as part of the AI ​​agent call, but it should conform to the definition provided in the AI ​​agent definition. An abstract tool definition is complete in terms of schema. In other words, the abstract definition defines how the tool is called, but does not precisely define how the tool is implemented. For example, consider a document summarization tool that takes a Portable Document Format (PDF) document as input and outputs a string. In this case, the tool can be implemented in various ways to achieve the same abstractly defined purpose. As another example, the tool may take a structure conforming to schema X as input (for example, specified as a JavaScript Object Notation (JSON) schema containing descriptions of each field in the schema, and the actual tool input must conform to that schema) and further have an output schema Y. In this case, at runtime, any tool implementation that conforms to the input schema X and output schema Y can be dynamically associated. Dynamic server-side tools are provided only as part of the AI ​​agent invocation.

[0028] In this specification, “invoking” an AI agent means starting or triggering the AI ​​agent to perform a specific task or action. That is, an AI agent is “invoked” to perform its function based on a given input or set of parameters. Invocation mechanisms may include user commands, specific events occurring within the system, or predefined schedules that prompt the activation of the AI ​​agent. Inputs to the AI ​​agent may include questions to answer, decisions to make, and sets of parameters to consider.

[0029] In all of the tool types described above, tool execution is performed directly by the agent loop and is not interrupted midway through the AI ​​agent request. The AI ​​agent request only returns a response to the caller to provide a final response or to update the user interface (UI), and typically does not pass control to the caller to change anything regarding tool usage. However, in some embodiments, it may be permissible for the caller to verify whether it is possible to invoke the tool at a particular point in time.

[0030] The same categories (i.e., static, call-specific, and dynamic) also exist for caller-side (i.e., client-side) tools. The difference is that caller-side tools delegate the execution of the tool to the caller within the context of a specific request (e.g., via bidirectional WebSocket communication with the server, or via multiple Hypertext Transfer Protocol (HTTP) requests that return state to the agent loop for restart or save it on the server side). This is because only the client can invoke such tools (e.g., file creation, screenshot capture). Therefore, this is usually useful when building agent-type behavior in the caller itself. In both the caller-side and server-side cases, the definition of a tool can be expressed by a schema (e.g., an annotated JSON schema) that describes the role of each parameter, acceptable values, the function of the tool itself, and when the tool should be used.

[0031] Tools may be undefined at all, defined only abstractly, or fully defined within the AI ​​agent definition as described above. Implementations of these tools may be injected as part of the agent process. In some embodiments, an AI agent implementing these tools can build other AI agents that can leverage the calling and server-side tools. For example, in some embodiments, a starting AI agent can invoke a tool-generating AI agent (i.e., "other" AI agents) which has its own agent loop and can attempt to analyze and modify the generated tool if the generated tool does not initially meet the requirements of the starting AI agent or does not properly satisfy the input / output (I / O) schema requested by the starting AI agent (if such a schema is requested). Alternatively, the starting AI agent can invoke a tool-generating AI agent and then attempt to utilize that tool within its own agent loop (and may modify the tool agentically using another agent if necessary).

[0032] In some embodiments, strong guardrails may be provided to mitigate hallucinations and prevent the AI ​​agent from taking undesirable paths. For example, if a user instructs the AI ​​agent to create a directory in the file system before creating a PDF file of an invoice, this sequence of operations can be enforced in the agent process (i.e., agent activities and custom agent loops). The AI ​​agent is provided with a valid execution trajectory. A “execution trajectory” refers to the specific sequence of actions and decisions that the AI ​​agent takes to complete a task. In other words, an execution trajectory substantially shows the path or route that the AI ​​agent takes from receiving input to generating output in the environment to achieve a goal, including adjustments or adaptations based on feedback and changing conditions along the way. A valid execution trajectory can be represented as a directed graph of valid operations and can be visualized, for example, using Mermaid. A valid execution trajectory can be softly enforced by adding to prompts or hard enforced by validating the validity of tool execution in agent loops.

[0033] To keep AI agents on a valid execution trajectory, the system can, for example, cause AI agent executions to fail, retry, and enforce a specific execution order. This is done by agent loops for hard enforcement or by verification AI agents for soft enforcement. This ensures that the nondeterminism of AI agents is controlled by strong guardrails. Without such guardrails, AI agents would be unstable.

[0034]

[0035] Figure 1 is an architectural diagram of a hyperautomation system 100 configured to perform agent automation and orchestration according to one embodiment of the present invention. As used herein, “hyperautomation” refers to an automation system that integrates components of process automation, agent automation, integration tools, and technologies that enhance the automation capabilities of work. Examples of these components include, but are not limited to, AI agents, AOP, and RPA robots.

[0036] Generally, in this specification, "AI agent" refers to AI-enhanced probabilistic automation that operates independently, behaves dynamically, makes decisions, performs actions, and acts adaptively. Such characteristics may result from the AI ​​agent utilizing a Large-Scale Language Model (LLM), which itself typically possesses probabilistic properties. "AOP" refers to automation that allows a user to describe an overall business process. AOP can be generated using an interface that allows the creation of business flowcharts written in BPMN, which is an Extensible Markup Language (XML) description of a business process. See, for example, Figure 5. "RPA robot" refers to automation that operates on a rule-based basis, behaves predictably, and makes deterministic decisions.

[0037] For example, in some embodiments, RPA can be used at the core of a hyperautomation system. Furthermore, in certain embodiments, automation capabilities can be extended by AI / machine learning (ML), process mining, analytics, agent automation, and / or other advanced tools. As the hyperautomation system learns processes, trains AI / ML models, and leverages analytics, more intelligent tasks can be automated, and computing systems within the organization (both those used by individuals and those operating autonomously) can become participants in the hyperautomation process. Some embodiments of the hyperautomation system enable users and organizations to efficiently and effectively discover, understand, and extend automation.

[0038] In such embodiments, the AI ​​agent "coexists" with the RPA robots that perform RPA and AOP. As described herein, the AI ​​agent is automation enhanced by AI skills, capable of operating independently, making dynamic decisions, performing actions, and adapting its performance. AI agents can dynamically leverage the tools available through these RPA robots to perform tasks such as document processing (see, for example, U.S. Patent Application Publication 2021 / 0097274), user interface (UI) automation (see, for example, U.S. Patents 10,654,166, 10,990,876, 11,080,548, 11,507,259, 11,733,668, and 11,748,069), and semantic copy and paste between source and target (see, for example, U.S. Patent 12,124,806 and U.S. Patent Application Publications 2023 / 0107316, 2023 / 0415338, and 2024 / 0220581). AI agents can dynamically select these tools and execute them as a pipeline.

[0039] Generally, agent automation is probabilistic automation performed by one or more AI agents. Agent automation expands an organization's automation potential by focusing not only on individual tasks but on entire end-to-end processes. A team of RPA robots directed by AI agents can enable one employee to accomplish a large number of tasks. Agent automation with AI agents gives managers room for guidance, allows physicians more time for patient care, gives developers the ability to fine-tune their work, gives engineers the freedom to innovate, and delivers a seamless, personalized experience to customers.

[0040] In some embodiments, agent automation can achieve a variety of technical effects, benefits, and gains. Agent automation improves memory usage by reducing the storage required for data storage and improves processor efficiency by reducing the number of calls and actions. Furthermore, agent automation can provide the ability to process gigabytes, terabytes, petabytes, or more of data that would be impossible for humans to process. In addition, dynamic decision-making can enable operation with fewer triggers and models. For example, a scenario that requires 100 actions with RPA alone can be significantly reduced (e.g., to 15 actions) using agent automation. Context grounding can also be used to constrain AI agents to a desired context. Thus, context grounding "constrains" the LLM to its relevant context.

[0041] An AI agent may have agent memory that evolutionarily stores user interactions, feedback, corrections, and solutions (e.g., dynamic user input from human-in-the-loop operations). “Human-in-the-loop” as used herein includes AI agents and RPA robots working collaboratively with a user and receiving dynamic, direct user input. In some embodiments, the AI ​​agent is not trained before deployment to a production environment but initially relies 100% on human-in-the-loop. As agent memory grows by processing human responses, the AI ​​agent becomes more autonomous, reducing the need for dynamic, direct human input and improving efficiency. This can be achieved through Retrieval Augmented Generation (RAG) or model fine-tuning (e.g., Supervised Fine-Tuning (SFT) or Low Rank Adaptation (LoRA)). Thus, as the AI ​​agent learns to handle more scenarios, it will attempt to autonomously and dynamically select and implement solutions using multiple available tools. Agent memory may also learn to make the AI ​​agent more efficient if it includes or derives more efficient solutions. For example, an AI agent can periodically process its agent memory to analyze patterns and achieve greater autonomy. Agent automation, once sufficiently trained, empowers the AI ​​agent to plan, work, and make decisions with minimal human supervision.

[0042] In this specification, “agent memory” refers to a dynamic cache (i.e., storage) system for managing escalations and tool calls. For example, if an AI agent encounters a problem during execution, it can request interaction or feedback from the user to resolve the problem, store / cached that interaction or feedback, and learn from it, thereby reducing the need for repeated human input. As one or more technical effects, benefits, and gains, agent memory improves efficiency by storing solutions to common problems and minimizing potentially costly tool calls. The collaborative operation of the AI ​​agent and agent memory makes it possible to “bend the curve” towards gradually reducing the need for human intervention as the AI ​​agent continuously learns through agent memory.

[0043] Generally, agent orchestration is implemented by a conductor application that executes one or more AOPs that utilize AI agents and RPA robots. In some embodiments, agent orchestration orchestrates AI agents (e.g., UiPath Agents®), third-party agents, RPA robots (e.g., UiPath Robots®), AOPs, and humans who execute agent workflows (e.g., where human approval is required). Thus, agent orchestration enables the automation, modeling, and monitoring of complex business processes from start to finish. Agent orchestration also offers a unique ability to integrally control RPA robots, AI agents, third-party agents, and humans across end-to-end agent workflows. Agent orchestration is beneficial for the successful scaling of agent automation.

[0044] AI agents for agent automation are AI model-based, as described above, enabling them to operate independently of humans and perform these agent automations. AI agents are also goal-oriented and make probabilistic decisions using context. Furthermore, AI agents are well-suited for ad-hoc tasks requiring high adaptability. AI agents learn how to work and improve over time. AI agents can use and select various tools to accomplish tasks, gather context, and perform actions (often via RPA robots, which the AI ​​agent uses as tools). In some embodiments, AI agents can build workflows and generate automations to be executed by RPA robots and / or other AI agents. For example, the creation and testing of automations can be accelerated by leveraging UiPath Autopilot™ for developers and other applications. For instance, an AI agent can use a designer application via an API to generate another AI agent or RPA workflow, which is then resolved by human-in-the-loop operations. If correct, that workflow can be deployed. AI agents can also have varying levels of autonomy controlled by agent orchestration.

[0045] The AI ​​agent executes an "agent loop," using the provided tools and context to generate a dynamic plan to achieve the goal according to instructions. Once the dynamic plan is generated, the AI ​​agent utilizes an efficient execution path for that plan. If there are two or more steps in the dynamic plan that can be executed in parallel, the AI ​​agent executes these steps in parallel based on available resources. Once each step is completed, the AI ​​agent retrieves its output and regenerates the next step or set of steps. In this way, the agent loop continues until the goal is achieved. Parallel execution of steps in the dynamic plan, and leveraging ecosystem tools and context grounding, are advanced features of agent orchestration.

[0046] In some embodiments, the AI ​​agent can also replan after each step. In other words, the initial plan is merely a suggestion. In more advanced scenarios, the AI ​​agent can also recognize that it is on the wrong path (for example, by pre-planning potential paths using a thought tree approach) and correct it retrospectively.

[0047] As described herein, RPA robots are rule-based, operate predictably, and make deterministic decisions. RPA robots are highly reliable and efficient and are suitable for routine tasks. RPA robots, along with AI agents, may utilize human-in-the-loop operations for exception handling. According to some embodiments, AI agents are more flexible, more abstract, and more self-deterministic than RPA robots and AOP. On the other hand, RPA robots are more stable, more concrete, and easier to control than AI agents and AOP. AOP processes fall between AI agents and RPA robots in terms of flexibility / stability, abstraction / concreteness, and self-determination / controllability.

[0048] As further explained in Figure 3, AI agents and RPA robots can discover and utilize each other as tools to accomplish tasks. AI agents and RPA robots can also access and utilize various applications (e.g., via APIs). Tools may be manually configured by developers, or they may be discovered and utilized by AI agents and RPA robots at runtime.

[0049] In some embodiments, AI agents, AOPs, and RPA robots work in collaboration with users (e.g., human-in-the-loop) to enable faster, more consistent, and more informed decision-making. Furthermore, the use of AI agents, AOPs, and RPA robots allows humans to accomplish more work because they undertake repetitive, monotonous, and ad-hoc tasks at a scale that would be difficult for humans to perform. When the AI ​​agent, AOP, or RPA robot encounters an exception, humans make the necessary decisions. Thus, humans can focus on their roles as supervisors, decision-makers, and organizational leaders.

[0050] AI models provide AI agents with the ability to reason, plan, generate, and make autonomous decisions. AI models may also be used by RPA robots in task-specific activities such as document processing and data analysis. AI models can be enhanced with business-specific content and context from a collection of enterprise context repositories, improving accuracy and results. AI models may be applied individually or in parallel depending on the complexity of the task. AI model selection can be based on RPA vendor model libraries, third-party models, and BYOM (Bring Your Own Model) options (see, for example, U.S. Patents 11,738,453 and 11,748,479).

[0051] The hyperautomation system 100 includes user computing systems such as a desktop computer 102, a tablet 104, and a smartphone 106. However, any user computing system such as a smartwatch, laptop computer, server, or IoT device can be used without departing from the scope of the present invention. Also, although three user computing systems are shown in Figure 1, any number of user computing systems can be used without departing from the scope of the present invention. For example, in some embodiments, tens, hundreds, thousands, or millions of user computing systems may be used. User computing systems may be actively used by the user or may operate automatically with little or no user input.

[0052] As disclosed herein, in some embodiments, there are three types of automation: (1) agent automation implemented by each AI agent, (2) RPA implemented by each RPA robot, and (3) composite automation that achieves a more complex overall task through a combination of AI agents and RPA robots. Automations 110, 112, and 114 may include, but are not limited to, those performed by RPA robots and / or AI agents. These may be performed individually or to achieve a larger composite automation. Other processes, such as listeners, may also be implemented. These processes may be standalone applications, subprocesses of other applications, parts of operating systems, other appropriate software and / or hardware, or any combination thereof. Without departing from the scope of the present invention, the logic of a process may be partially or completely implemented by physical hardware.

[0053] Each user computing system 102, 104, and 106 has automations 110, 112, and 114 running on it, respectively, which are implemented by RPA robots, AI agents, etc. In some embodiments, automations 110, 112, and 114 may be stored remotely (e.g., on a server 130 or a database 140), accessed via the network 120, and read and executed by RPA robots and / or AI agents. The database 140 may store structured data and / or unstructured data, but the former is usually required in RPA. RPA automation may exist as scripts (e.g., XML, XAML, etc.) or compiled as machine-readable code (e.g., dynamic link libraries). In the case of AI agents, agent automation may be generated based on, for example, a plain text description of a desired target.

[0054] The listener monitors and records data on user operations in each computing system and the operation of unattended computing systems, and transmits it to the core hyperautomation system 120 via a network (e.g., LAN, mobile communication network, satellite communication network, internet, or any combination thereof). The data may include, but is not limited to, clicked buttons, mouse movement positions, text entered into fields, window minimization and opening of other windows, and applications associated with windows. In certain embodiments, data from the listener may be transmitted periodically as part of heartbeat messages. It may also be transmitted after a predetermined amount of data has been collected, or after a predetermined amount of time has elapsed, or both. One or more servers, such as server 130, receive the data from the listener and store it in a database 140 or the like.

[0055] If automations 110, 112, and 114 are RPA, they execute logic developed within the workflow at design time. A workflow consists of a set of steps defined as “activities,” which are executed in sequence or other logical flow. Each activity may include actions such as clicking a button, reading a file, or writing to a log panel. In some embodiments, workflows can be nested or embedded.

[0056] In some embodiments, long-running RPA workflows are master projects that support service orchestration, human-in-the-loop, and long-running transactions in unattended environments (see, for example, U.S. Patent No. 10,860,905). Human-in-the-loop functions when human input (dynamic direct user input) is required for exception handling, approval, or verification. In this case, process execution is paused and the RPA robot is released until human processing is complete.

[0057] Long-running workflows support workflow partitioning through persistent activities and orchestrate human-in-the-loop tasks and RPA robot tasks in combination with invocation processes and non-user interaction activities. In some embodiments, multiple computing systems may participate in the logic execution of the long-running workflow. Long-running workflows may run within a session for faster execution. In some embodiments, background processes, including activities that make API calls, are orchestrated and run within the long-running workflow session. These activities may be initiated by invocation process activities. Processes involving user interaction may run in a user session by starting a job from a conductor activity. Users may interact through form-filling tasks within the conductor. Activities may include causing the RPA robot to wait for form completion and then resume the long-running workflow.

[0058] One or more of the automations 110, 112, and 114 communicate with the core hyperautomation system 120. In some embodiments, the core hyperautomation system 120 runs conductor applications on one or more servers, such as server 130. Although one server 130 is shown for illustrative purposes, multiple servers can be used in a close-proximity or distributed architecture without departing from the scope of the invention. For example, multiple servers may be provided for conductor functions, AI / ML model provisioning, authentication, governance, etc. In some embodiments, the core hyperautomation system 120 may be configured as part of a public cloud, private cloud, or hybrid cloud architecture. In certain embodiments, the core hyperautomation system 120 hosts multiple software-based servers on one or more computing systems, such as server 130. In some embodiments, servers such as server 130 may be implemented by virtual machines (VMs).

[0059] In some embodiments, one or more of the automations 110, 112, and 114 can invoke AI / ML models 132 that are located on or accessible by the core hyperautomation system 120. These models are trained to perform a variety of tasks. For example, AI / ML models 132 may include models that perform application version detection, computer vision (CV), optical character recognition (OCR), user interface (UI) descriptor generation, suggestion of the next activity or sequence in an RPA workflow, semantic matching, natural language processing (NLP), and generation or modification of code and / or RPA workflows. AI / ML models can be trained using labeled data that includes data source elements such as web pages, forms, scanned documents, application interfaces, and screens, as well as existing RPA workflows, screenshots of various application screens and their corresponding UI elements, and a UI object library. AI / ML models 132 are trained to achieve a desired confidence threshold while avoiding overfitting. In general, UI elements, UI descriptors, applications, and application screens can be considered UI objects.

[0060] The AI / ML model 132 can be trained for any purpose without departing from the scope of the present invention, which will be described in more detail later. Multiple AI / ML models 132 may be linked in series, parallel, or a combination thereof in some embodiments to provide collaborative output. The AI / ML model 132 may perform or assist with CV, OCR, document processing and / or understanding, semantic learning and / or analysis, analytical prediction, process discovery, task mining, testing, automated RPA workflow generation, sequence extraction, cluster detection, speech-to-text conversion, NLP, semantic matching, etc. However, any number and / or types of AI / ML models can be used without departing from the scope of the present invention. Using multiple AI / ML models makes it possible to get a bigger picture of what is happening on a particular computing system, for example. For example, one model may perform OCR, another model may detect buttons, and yet another model may compare sequences. Patterns may be determined individually by a single AI / ML model, or they may be determined jointly by multiple AI / ML models. In some embodiments, one or more AI / ML models are locally located on at least one of the computing systems 102, 104, and 106.

[0061] In some embodiments, multiple AI / ML models 132 may be used. Each AI / ML model 132 is an algorithm (or model) executed on data, and for example, the AI / ML model itself may be a deep learning neural network (DLNN) of artificial "neurons" trained on training data. In some embodiments, the AI / ML model 132 has multiple layers that perform various functions such as statistical modeling (e.g., Hidden Markov Models (HMMs)) and may utilize deep learning techniques (e.g., Long Short-Term Memory (LSTM) deep learning, encoding of previous hidden states, etc.) to perform the desired function.

[0062] In some embodiments, the hyper-automation system 100 may provide four main functional groups: (1) discovery, (2) automation construction, (3) management, and (4) engagement. Automation (e.g., executed on user computing systems, servers, etc.) may be performed, for example, in some embodiments, by RPA robots, AOP, or AI agents, and may provide any of the functions described herein. For example, RPA robots may include attendant robots, unattendant robots, and / or test robots. Attendant robots assist with tasks in collaboration with the user (e.g., via UiPath Assistant®). Unattendant robots operate independently of the user and may run in the background without the user's knowledge. Test robots execute test cases against an application or RPA workflow. In some embodiments, test robots may run in parallel on multiple computing systems.

[0063] Discovery capabilities can identify different automation opportunities in business processes and provide automated recommendations. Such capabilities may be implemented by one or more servers, such as server 130. In some embodiments, discovery capabilities may include providing an automation hub, process mining, task mining, and / or task capture. An automation hub (e.g., UiPath Automation Hub®) may provide a mechanism for managing automation deployments with visibility and control. For example, automation ideas may be crowdsourced from employees via a submission form. Feasibility and return on investment (ROI) calculations for automating these ideas may be provided, documentation for future automations may be collected, and collaboration may be provided to accelerate the process from automation discovery to build.

[0064] Process mining (e.g., via UiPath Automation Cloud® and / or UiPath AI Center®) refers to the process of collecting and analyzing data from applications (e.g., ERP applications, CRM applications, email applications, call center applications, etc.) to identify what end-to-end processes exist within an organization, how to effectively automate them, and what impact that automation will have. This data may be collected, for example, by listeners from user computing systems 102, 104, and 106 and processed by a server such as server 130. In some embodiments, one or more AI / ML models 132 may be used for this purpose. This information may be exported to an automation hub to expedite implementation and avoid manual data transfer. The objective of process mining may be to increase business value by automating processes within an organization. Examples of process mining goals include, but are not limited to, increased profits, improved customer satisfaction, regulatory and / or contractual compliance, and improved employee efficiency.

[0065] Task mining (e.g., via UiPath Automation Cloud® and / or UiPath AI Center®) identifies and aggregates workflows (e.g., employee workflows), then applies AI to reveal patterns and variations in daily tasks, and scores such tasks in terms of ease of automation and potential savings (e.g., time and / or cost savings). In some embodiments, one or more AI / ML models 132 may be used to reveal recurring task patterns in the data. Then, repetitive tasks suitable for automation can be identified. In some embodiments, this information is initially provided by a listener and can be analyzed on a server of the core hyperautomation system 120, such as server 130. The results of task mining (e.g., XAML process data) can be exported to a process document or a designer application such as UiPath Studio® to more quickly create and deploy automations. In some embodiments, task mining may include taking screenshots with user interactions (e.g., mouse click locations, keyboard inputs, application windows and graphical elements the user was interacting with, timestamps of interactions, etc.), collecting statistical data (e.g., execution time, number of actions, number of text inputs, etc.), editing and annotating screenshots, specifying the types of actions to be recorded, and so on.

[0066] Task capture (e.g., via UiPath Automation Cloud® and / or UiPath AI Center®) automatically documents the processes that a user works on, or provides a framework for unattended processes. Such documentation may include automated tasks in the form of Process Definition Documents (PDDs), skeletal workflows, capture of actions for each part of the process, recording of user actions, automatic generation of comprehensive workflow diagrams including details of each step, Microsoft Word® documents, XAML files, etc. In some embodiments, the build-ready workflow can be directly exported to a designer application such as UiPath Studio®. In some embodiments, task capture can simplify the requirements gathering process for both subject matter experts describing the process and members of a Center of Excellence (CoE) that deliver production-quality automation.

[0067] Automation can be built via designer applications (e.g., UiPath Studio®, UiPath StudioX®, or UiPath Studio Web®). For example, developers in the RPA development facility 150 can use the designer application 154 of the computing system 152 to build and test agent automation, RPA, AOP, and / or combined automation for various applications and environments such as web, mobile, SAP®, and virtualized desktops. Developers can also build AOP. For example, developers can create automations that are executed by RPA robots, AI agents, AOP, combinations thereof, etc. API integrations for various applications, technologies, and platforms can be provided. Predefined activities, drag-and-drop modeling, and workflow recorders can facilitate automation with minimal coding. Document understanding capabilities can be provided via drag-and-drop AI skills for data extraction and interpretation, which invoke one or more AI / ML models 132. Such automations can handle virtually any document type and format, including tables, checkboxes, signatures, and handwritten text. If data is validated or exceptions are handled, this information can be used to retrain the corresponding AI / ML model, potentially improving its accuracy over time.

[0068] The designer application 152 may be designed to invoke one or more trained AI / ML models 132 on the server 130 and / or one or more generated AI models 172 in a cloud environment via the network 120 (e.g., LAN, mobile communication network, satellite communication network, internet, or any combination thereof) to support the automated development process. In some embodiments, one or more AI / ML models may be packaged in the designer application 152 or stored locally on the computing system 150.

[0069] In some embodiments, one or more of the designer application 152 and AI / ML models 132 may be configured to use an object repository stored in a database 140. See, for example, U.S. Patent No. 11,748,069, which is incorporated in its entirety herein by reference. Generally, an object repository is a storage mechanism used by automation for images, text, semantic data, taxonomic associations, ontological associations, UI objects, etc. For example, an object repository may include a library of UI objects that can be used to develop RPA workflows via the designer application 152. The object repository may be used for UI automation to add UI descriptors to activities in the workflow of the designer application 152. In some embodiments, one or more of the AI / ML models 132 may generate new UI descriptors and add them to the object repository in the database 140.

[0070] Once the automation is complete in the designer application 152, it can be published on the server 130 and pushed to computing systems 102, 104, 106, etc. For example, as new UI descriptors are created and / or existing UI descriptors are modified, a global repository of a shareable and collaborative UI object library for all automations can be built. With respect to the object repository, taxonomies and ontologities can be used. A taxonomy is a hierarchical structure of subcategories. An ontology is a formal representation of a knowledge domain, including concepts, characteristics, and the relationships between them. In an ontology, the relationships between categories are not necessarily hierarchical, and ontological relationships can span multiple screens of an application.

[0071] Integrated services can, for example, enable developers to seamlessly combine UI automation and API automation. Automation, like any type of automation described herein, can be built to require APIs or to span both API and non-API applications and systems. A repository (e.g., UiPath Object Repository®) or marketplace (e.g., UiPath Marketplace®) for pre-built automation templates and solutions may be provided to enable developers to automate a wide variety of processes more quickly. Thus, when building automation, the hyperautomation system 100 may provide a user interface, development environment, API integration, pre-built and / or custom-built AI / ML models, development templates, an integrated development environment (IDE), and advanced AI capabilities. In some embodiments, the hyperautomation system 100 may enable the development, deployment, management, configuration, monitoring, debugging, and maintenance of RPA robots and AI agents, which may provide automation for the hyperautomation system 100.

[0072] In some embodiments, components of the hyperautomation system 100, such as a designer application and / or an external rule engine, provide assistance in managing and enforcing governance policies to control the various functions provided by the hyperautomation system 100. Governance is the ability of an organization to implement policies to prevent users from developing automations (e.g., RPA robots and / or AI agents) that have the ability to take actions that could harm the organization, such as violating the EU General Data Protection Regulation (GDPR), the US Health Insurance Portability and Accountability Act (HIPAA), or the terms of use of third-party applications. Because developers could otherwise create automations that violate privacy laws, terms of use, etc., while running the automation, in some embodiments, access control and governance restrictions are implemented at the robot level and / or robot design application level. This can provide an additional level of security and compliance to the automation process development pipeline by preventing developers from relying on unauthorized software libraries that could introduce security risks or operate in a manner that violates policies, regulations, privacy laws, and / or privacy policies. For example, see U.S. Patent No. 11,733,668, which is incorporated in its entirety herein by reference.

[0073] The management functions can provide management, deployment, and optimization of automation across the entire organization. In some embodiments, the management functions may include orchestration, test management, AI capabilities, and / or insights. The management functions of the hyperautomation system 100 may also function as an integration point with third-party solutions and applications for automation applications and / or RPA robots. The management capabilities of the hyperautomation system 100 may include, but are not limited to, facilitating the provisioning, deployment, configuration, queuing, monitoring, logging, and interoperability of RPA robots and / or AI agents.

[0074] Conductor applications such as UiPath Orchestrator® (which, in some embodiments, may be offered as part of UiPath Automation Cloud®, or as a cloud-native single-container suite on-premises, in a VM, in a private or public cloud, in a Linux® VM, or via UiPath Automation Suite®) provide orchestration capabilities for deploying, monitoring, optimizing, scaling, and securing RPA robots and / or AI agents. A test suite (e.g., UiPath Test Suite®) may provide test management for monitoring the quality of deployed automations. A test suite can facilitate test planning and execution, requirements fulfillment, and defect traceability. A test suite may include comprehensive test reports.

[0075] Analytics software (e.g., UiPath Insights®) can track, measure, and manage the performance of deployed automations. Analytics software can align automation operations with specific key performance indicators (KPIs) and strategic outcomes for the organization. Analytics software can present results in a dashboard format to enhance human user understanding.

[0076] A data service (e.g., UiPath Data Service®) can store data in a single, scalable, and secure location, for example, a database 140, with a drag-and-drop storage interface. In some embodiments, it can provide low-code or no-code data modeling and storage for automation while ensuring seamless access, enterprise-grade security, and data scalability. AI functionality can be provided by an AI Center (e.g., UiPath AI Center®), which facilitates the integration of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options can make such functionality accessible even to non-data scientists. Deployed automation (e.g., an RPA robot) can call AI / ML models, such as AI / ML model 132, from the AI ​​Center. The performance of AI / ML models can be monitored and trained and improved using human-validated data, such as that provided by a data review center 160. A human reviewer may provide labeled data to the core hyperautomation system 120 via a review application 152 on the computing system 154. For example, the human reviewer may verify that the predictions by the AI / ML model 132 and / or the generative AI model 172 are accurate and provide corrections if not. The human reviewer may also provide dynamic direct user input to the AI ​​agent (e.g., within the scope of human-in-the-loop operation), and the responses and corrections provided by the human reviewer may be used to train the LLM used by the AI ​​agent to be more accurate. In other words, this dynamic input may be stored as training data for retraining the AI / ML model 132 and / or the generative AI model 172, and may be stored in a database, for example, database 140. The AI ​​center may then schedule and execute training jobs to train a new version of the AI / ML model using the training data.Both positive and negative examples are stored and can be used to retrain the AI / ML model 132 and / or the generative AI model 172.

[0077] The engagement feature brings humans and automation together as a single team to enable seamless collaboration on a desired process. Low-code applications can be built (for example, via UiPath Apps®) and, in some embodiments, can connect browser tabs and legacy software, and even those lacking APIs. For example, applications can be quickly created through a web browser using a rich drag-and-drop control library. Applications can connect to a single automation or multiple automations.

[0078] An action center (e.g., UiPath Action Center®) provides a simple and efficient mechanism for handing over processes from automation to humans and vice versa. Humans can provide approvals or escalations, handle exceptions, etc. Automation can then execute the automated functions of a given workflow.

[0079] A local assistant may be provided as a launchpad for the user to launch automations (e.g., UiPath Autopilot®). Such an assistant may also provide semantic cut-and-paste functionality (e.g., UiPath Clipboard AI®). See, for example, U.S. Patent No. 12,124,806 and U.S. Patent Application Publications 2023 / 0107316, 2023 / 0415338, and 2024 / 0220581. This functionality may be provided, for example, within a tray provided by the operating system, allowing the user to interact with RPA robots and RPA robot-driven applications on their computing system. The interface may list automations approved for a given user and allow the user to run them. These may include ready-to-use automations from an automation marketplace, an internal automation store in an automation hub, etc. When automations are executed, they may run as local instances in parallel with other processes on the computing system, so that the user can use the computing system while the automations are performing their actions. In certain embodiments, the assistant is integrated with a task capture function, so that the user can document processes that will soon be automated from the assistant launchpad.

[0080] In some embodiments, the hyper-automation system 100 may provide end-to-end measurement and governance of automation programs of any size. As described above, analytics may be used to understand the performance of automation (e.g., via UiPath Insights®). Data modeling and analytics using any combination of available business metrics and operational insights may be applied to various automation processes. Custom-designed and pre-built dashboards enable data visualization across desired metrics, discovery of new analytical insights, tracking of performance metrics, discovery of ROI for automation, telemetry monitoring on user computing systems, detection of errors and anomalies, and debugging of automation. An automation management console (e.g., UiPath Automation Ops®) may be provided to manage automation throughout the entire automation lifecycle. Organizations can control how automation is built, what users can do with it, and which automations users can access.

[0081] The hyper-automation system 100, in some embodiments, provides an iterative platform. Processes can be discovered, automations can be built, tested, and deployed, performance can be measured, automation use can be easily provided to users, feedback can be obtained, AI / ML models can be trained and retrained, and the process can be repeated. This promotes a more robust and effective set of automations.

[0082] In some embodiments, as described above, generative AI models are used. For example, an AI agent utilizes a generative AI model. Generative AI models can generate various types of content, such as text, images, audio, and synthetic data. Various types of generative AI models can be used, including but not limited to large-scale language models (LLMs), generative adversarial networks (GANs), diffusion models, flow-based models, variational autoencoders (VAEs), and transformers. For example, in the case of LLMs, natural language processing (NLP) models such as word2vec, BERT, GPT-3, and ChatGPT may be used in some embodiments to facilitate semantic understanding and provide more accurate and human-like responses. These models may be part of an AI / ML model 132 hosted on server 130. For example, a generative AI model may be trained on a large text information corpus to perform semantic understanding, understand the properties of things present on a screen from text, and automatically generate code. An AI agent may use such a generative AI model. In certain embodiments, generative AI models 172 provided by existing cloud ML service providers such as OpenAI®, Google®, Amazon®, Microsoft®, IBM®, Nvidia®, and Meta® may be used and trained to provide such functionality. In generative AI embodiments where the generative AI model 172 is remotely hosted, the server 130 may be configured to integrate with a third-party API so that the server 130 can send requests containing the necessary input information to the generative AI model 172 and receive responses (e.g., semantic matching of fields between application versions, classification of application types on a screen, responses to natural language queries from the user, etc.). Such embodiments may provide a more advanced and sophisticated user experience and may provide access to state-of-the-art NLP and other ML capabilities offered by these companies.

[0083] One form of generative AI model in some embodiments is the use of transfer learning. In transfer learning, a pre-trained generative AI model, such as a Language Language Model (LLM), is fine-tuned for a specific task or domain. This allows the LLM to leverage the knowledge it has already learned during its initial training and adapt it to a specific application. In the case of an LLM, the pre-training stage typically involves training the LLM on a large text corpus consisting of billions of words. During this stage, the LLM learns the relationships between words and phrases, enabling it to generate consistent, human-like responses to text-based input. The output of this pre-training stage is an LLM with a high degree of understanding of underlying patterns in natural language.

[0084] In the fine-tuning phase, a pre-trained LLM is adapted to a specific task or domain by training the LLM on a smaller dataset specific to that task. For example, in some embodiments, the LLM may be trained to analyze specific types or combinations of data sources to improve its accuracy regarding their contents. This data may include, but is not limited to, prompt tuning or instruction tuning, where the model is specifically trained to better understand and follow certain types of instructions or prompts. This improves its ability to perform a particular task when given appropriate instructions. Such information may be provided as part of the training data, allowing the LLM to focus on these areas and learn to more accurately identify the data elements contained therein. Through fine-tuning, the LLM can learn subtle differences in a task or domain, such as specific vocabulary and syntax used in that domain, without requiring the large amount of data needed to train the LLM from scratch. By leveraging the knowledge learned in the pre-training phase, a fine-tuned LLM can achieve state-of-the-art performance for a particular task with relatively small amounts of training data.

[0085] LLM can use vector databases. Vector databases index, store, and provide access to structured or unstructured data (e.g., text, images, time series data, etc.) along with their vector embeddings. Text-like data can be tokenized, with single characters, words, or sequences of words parsed from the text as tokens. These tokens are then "embedded" in vector embeddings, which are numerical representations of this data. Vector databases enable LLM to quickly and extensively find and retrieve similar objects in a production environment, which is not possible with manual processes.

[0086] AI and ML enable the numerical representation of unstructured data in vector embeddings without losing their semantic meaning. A vector embedding is a sequence of numbers that describes the features of the data object it represents. Similar objects are grouped together in the vector space. In other words, the more similar the objects, the closer the vector embeddings representing them are. Similar objects can be found using vector search, similarity search, or semantic search. The distance between vector embeddings can be calculated using a variety of techniques, including but not limited to squared Euclidean distance or L2-squared distance, Manhattan distance or L1 distance, cosine similarity, dot product, Hamming distance, etc. It may be beneficial to choose the same metrics used to train the AI / ML model.

[0087] Vector indexing can be used to organize vector embeddings so that data can be retrieved efficiently. Calculating the distance between one vector embedding and all other vector embeddings in a vector database using the k-nearest neighbors (kNN) algorithm can be computationally expensive, as the required computation increases linearly (O(n)) with respect to the number of dimensions and data points when the number of data points is large. Finding similar objects using an approximate nearest neighbors (ANN) approach is more efficient. Similar objects can be found much faster because the distances between vector embeddings are pre-calculated, and similar vectors are organized and stored close to each other (e.g., in clusters or graphs). This process is called "vector indexing." ANN algorithms that may be used in some embodiments may include, but are not limited to, clustering-based indexing, proximity-graph-based indexing, tree-based indexing, hash-based indexing, and compression-based indexing.

[0088] Figure 2 shows a portion of the coupling capability 200 of an AI agent 210 and an RPA robot 220 according to one embodiment of the present invention. The AI ​​agent 210 is configured to process natural language instructions, achieve expected goals therefrom, execute through dynamic decision-making or dynamic flow control with self-correcting capabilities, store information in long-term memory to evaluate its own performance, and learn from human-in-the-loop and its own performance during execution. The RPA robot 220 may be leveraged by the AI ​​agent 210 to respond to triggers (e.g., from a conductor application such as UiPath Orchestrator®), respond based on context (i.e., the RPA robot 220 may take information from the context to perform deterministic steps such as updating a document based on the acquired context information; alternatively, the agent 210 may use the acquired context to update a dynamic plan and perform the next steps to complete the goal according to the instructions), leverage AI models (e.g., CV models, document processing models, speech-to-text models, OCR models, etc.), leverage RPA tools (e.g., utilizing tools available within the RPA ecosystem such as completed automations, workflows within automations, integrated service connector calls for third-party and first-party services, RPA designer application activities, LLM calls, etc.), and perform actions that the RPA robot can perform based on input from the AI ​​agent (i.e., using the RPA robot as a tool). AI agent 210 can also update its own memory, update its plan to achieve goals according to instructions, perform self-assessment and learning from its actions, self-correct when it encounters obstacles, and take action to escalate to a human when assistance is needed.

[0089] As described above, in some embodiments, agent automation can achieve a variety of technical effects, advantages, and benefits. Agent automation improves memory usage by reducing the amount of storage required for data and improves processor efficiency by reducing the number of calls and actions. Agent automation also potentially provides the ability to process gigabytes, terabytes, petabytes, or more of data that would be impossible for a human to perform mentally or manually. It also potentially enables the use of fewer triggers and models through dynamic decision-making. While RPA alone may require 100 actions in one example scenario, this can be drastically reduced (e.g., to as few as 15 actions) by using agent automation. Context grounding can also be used to tie the AI ​​agent to the desired context for agent automation. This "constrains" the LLM to the relevant context.

[0090] In this specification, “context grounding” refers to a methodology that improves models such as LLMs by integrating enterprise-specific information with pre-trained knowledge, enabling accurate responses to specialized or up-to-date queries. In some embodiments, context grounding uses external data to augment LLM responses, obtaining responses that the LLM would not inherently know and answering queries based on the provided context. For example, proprietary industry jargon and complex document structures can be challenges in ensuring effective retrieval and semantic matching, but context grounding addresses such challenges by providing precise chunking of documents, ensuring that relevant information (e.g., derived from proprietary industry jargon and complex document structures) is passed to the LLM without noise. As an additional example, context grounding provides enhanced extraction and retrieval techniques (e.g., adapted to proprietary industry jargon and complex document structures) that are tailored to diverse industries and applications, thereby improving LLM responses.

[0091] Figure 3 shows a pool 300 of AOPs, AI agents, RPA robots, and applications according to one embodiment of the present invention. The AOP pool 310 includes AOPs 1, 2, ..., P which implement business processes. As described above, AOPs can be implemented as BPMNs executed by an AOP execution engine such as Temporal®. AOPs can utilize AI agents and / or RPA robots to execute parts of their business processes.

[0092] The AI ​​agent pool 320 includes AI agents 1, 2, ..., I, trained to perform various tasks such as investigating claims, seeking solutions with human employees, and summarizing policies and technical specifications. The RPA robot pool 330 includes RPA robots 1, 2, ..., J, which perform various automations such as UI automation, semantic matching automation, and form entry automation. The application pool 340 includes applications 1, 2, ..., K, with which the AI ​​agents and / or RPA robots can interact. For example, applications may include CRM applications, invoicing applications, payroll applications, banking applications, web applications, legacy system applications, word processing applications, spreadsheet applications, email applications, etc. The AI ​​agents, RPA robots, and applications may reside on a single computing system or on multiple or numerous computing systems. The AOP typically resides in the cloud or on another server side and, in some embodiments, may reside on the same computing system as the conductor application 350.

[0093] AOP can trigger or invoke AI agents and RPA robots via the conductor application 350. AI agents and RPA robots can also trigger or invoke each other via the conductor application. For example, to invoke an RPA robot, an AI agent may make a "Start Job" call within the conductor application 350. Note that RPA robots are deployed as automation controlled by the conductor application 350. AI agents and RPA robots can also trigger or invoke specific applications. For example, through information obtained from human-in-the-loop actions, an AI agent may dynamically learn which RPA robots, other AI agents, and / or applications to trigger or invoke to accomplish a task. For example, an AI agent may learn to trigger an RPA robot via the conductor application 350 to fill out and submit a web form. The AI ​​agent may also learn to open Microsoft Excel®, enter form information into the appropriate tabs, and open and update a payroll application, etc. Furthermore, the AI ​​agent may learn to call or trigger an email resolution AI agent via the conductor application 350 to contact a human customer service representative of the bank in the event of a problem. In some embodiments, the technical effects, benefits, and advantages may be similar to those described above with respect to Figures 1 and 2.

[0094] AI agents and RPA robots may belong to a tenant in order to find each other. A designer application may call a conductor to obtain a list of available RPAs. In some embodiments, there are three ways to obtain the capabilities of an automation: (1) by providing a description of what the automation does when a user creates a workflow in the designer application; (2) by using AI agents and ML techniques to generate a summary of what a given workflow does; or (3) by a developer describing what the automation does within the designer application. A conductor application may also have a list of which applications are available for a particular AI agent and RPA robot. In other words, descriptions of available AI agents, RPA robots, and / or applications are derived or provided by the AI ​​agent, ML techniques, or user.

[0095] Figures 4A and 4B show an example of an agent service interface 400 according to one embodiment of the present invention. Referring to Figure 4A, the agent answers questions regarding a policy document provided within context grounding. The agent instruction pane 410 contains a natural language description entered by the user about what the AI ​​agent is intended to do. User prompt 420 allows the developer to enter content for the user prompt in the content field 422, if necessary. The tool dropdown 430 allows the developer to select the tools the AI ​​agent will use, such as using an API for the application or calling an RPA robot to perform RPA.

[0096] The context dropdown 440 allows developers to configure context grounding for the AI ​​agent. The context configuration pane 442 allows developers to provide a description via the description field 444 and an Elastic Common Schema (ECS) index via the ECS index field 446 for a specific policy document that contains information such as contracts, conditions, and what to do. Developers can also add additional contexts 450 to further reinforce the context grounding. The option to escalate to a human can be configured via the dropdown 460.

[0097] The query field 470 allows the user to provide a query that the AI ​​agent will respond to. The AI ​​agent executes the query when the user clicks the execute button 480. Moving to Figure 4B, the results of the AI ​​agent's execution are then displayed in the execution pane 490 as the AI ​​agent retrieves and outputs them.

[0098] Figure 5 shows an example of an AOP development interface 500 according to one embodiment of the present invention. The AOP development interface 500 includes AOP components 510, an AI agent 520, and an RPA 530 that can be selected by the user when developing a business process. These are selected and dragged onto a canvas 540, where the user can manually develop the AOP. In this example, a credit check is implemented by calling an AI agent to retrieve customer data from a database and analyze the customer data to determine the customer type (e.g., very likely to pay, likely to default, frequently between unemployment and employment, etc.). The type is then provided to an RPA robot that considers this information when performing the credit check. Alternatively, the AOP developer can enter a description of the business process in field 550 and click the generate button 560. This text is provided to the LLM, which understands the requested business process and attempts to automatically create an AOP workflow. The AOP developer can then edit the AOP workflow as needed.

[0099] Figure 6 shows an example of an RPA development interface 600 according to one embodiment of the present invention. The RPA development interface 600 includes RPA components 610 that can be selected by the user when developing an RPA workflow. These can be selected and dragged onto the canvas 620. Alternatively, the RPA developer can enter a description of the RPA in the field 630 and click the generate button 640. This text is provided to the LLM, which understands the requested business process and attempts to automatically create an RPA workflow. The developer can then edit the RPA workflow as needed. Note that the functions shown and described with respect to Figures 4A, 4B, 5, and 6 may be provided in a single designer application in some embodiments.

[0100] Figure 7 shows an end-to-end AI agent, RPA robot, and AOP development and deployment system 700 according to one embodiment of the present invention. A designer application 710 enables developers to design AOP, AI agents, and RPA workflows. Once these are tested and validated, they are packaged and published to the automation database 720.

[0101] The conductor application 730 manages these automations, as well as the deployment of AOPs, AI agents, and RPA robots. When a human user or software process 732 requests the execution of an AOP, the conductor application 730 sends a start job request to the AOP engine 740, which then selects and starts the appropriate automation from among the AOPs 742. While executing an AOP 742, it may encounter a step that is to be implemented by an AI agent 750 or an RPA robot 760. In this case, the AOP engine 740 pauses the execution of the AOP workflow and sends a request to the conductor application 730 to send a start job request to the appropriate AI agent 750 or RPA robot 760 to execute that step.

[0102] If an AI agent is requested, the conductor application 730 sends a start job request to the appropriate AI agent 750. This request may include natural language text or other information provided to the conductor application 730 from the AOP engine 740. The AI ​​agent 750 then performs this step by executing the LLM 752 to assist in the execution of the task. The AI ​​agent 750 then sends information related to the task (e.g., requested information, instructions that the step is complete, instructions that the step has failed, etc.) to the conductor 730, which then provides this information to the AOP engine 740. The AOP engine 740 then resumes its operation.

[0103] When an RPA robot is requested, the conductor application 730 sends a start job request to the appropriate RPA robot 760. The RPA robot 760 then executes the requested RPA 762. The RPA robot 760 then sends task-related information (e.g., requested information, instructions that a step has been completed, instructions that a step has failed, etc.) to the conductor 730, which provides this information to the AOP engine 740. The AOP engine 740 then resumes its operation.

[0104] In some cases, human intervention may be required by AOP742, AI agent 750, or RPA762. In this case, AOP engine 740, AI agent 750, or RPA robot 760 contacts a human 770 for the human-in-the-loop portion of the automation. After the human completes the task, AOP engine 740, AI agent 750, or RPA robot 760 resumes the automated portion of the automation.

[0105] Figure 8 is an architectural diagram showing an agent automation and RPA system 800 according to one embodiment of the present invention. In some embodiments, the agent automation and RPA system 800 is part of the hyperautomation system 100 in Figure 1. The agent automation and RPA system 800 includes a designer 810 that enables developers to design automations for AI agents and RPA robots (e.g., workflows, natural language instructions for AI agents, context grounding, tool configuration, etc.). The designer 810 can provide solutions for application integration, as well as solutions for automating third-party applications, managed information technology (IT) tasks, and business IT processes. The designer 810 can facilitate the development of automation projects, which are graphical representations of business processes. In short, the designer 810 facilitates the development and deployment of automations for RPA robots and AI agents. In some embodiments, the designer 810 may be an application that runs on the user's desktop, an application that runs remotely within a VM, a web application, etc.

[0106] Automation projects enable the automation of rule-based processes by allowing developers to control the execution order and relationships of the custom set of steps developed within the workflow, i.e., the "activities" described above. One commercial example of an embodiment of Designer 810 is UiPath Studio®. Each activity may include actions such as clicking a button, reading a file, or writing to a log panel. In some embodiments, workflows may be nested or embedded.

[0107] Workflow types may include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. Sequences are particularly suitable for linear processes and can allow a flow from one activity to another without complicating the workflow. Flowcharts are particularly suitable for more complex business logic and can allow for the integration of decision-making and the connection of activities in more diverse ways through multiple branching logic operators. FSMs are particularly suitable for large-scale workflows. FSMs can use a finite number of states in their execution, which are triggered by conditions (i.e., transitions) or activities. Global exception handlers are particularly suitable for determining workflow behavior and debugging processes when execution errors are encountered.

[0108] Once workflows and / or other configurations for AI agents are developed in Designer 810, the execution of business processes is orchestrated by Conductor 820, which orchestrates one or more robots 830, one or more AI agents 850, and / or one or more AOPs 870 that execute the workflows developed in Designer 810. One commercial example of an embodiment of Conductor 820 is UiPath Orchestrator®. Conductor 820 facilitates the creation, monitoring, and management of resources in the environment. Conductor 820 can function as an integration point with third-party solutions and applications. As described above, in some embodiments, Conductor 820 may be part of the core hyperautomation system 120 in Figure 1.

[0109] It should be noted that the RPA robot 830 can operate independently of deterministic processes. The AI ​​agent 850 and AOP 870 can also operate independently (e.g., of non-deterministic processes), or they can utilize the RPA robot 830 and / or other AI agent 850 as tools to achieve part of their agent automation. The AI ​​agent 850 can drive a composite automation that utilizes both the RPA robot 830 and the AI ​​agent 850, or vice versa, and the AOP 870 can include such a composite automation.

[0110] The conductor 820 manages a fleet of robots 830 and AI agents 850, and can connect and run the RPA robots 830 and AI agents 850 from a central point (e.g., as required by an AOP engine implementing AOP). The types of RPA robots 830 that can be managed may include, but are not limited to, attendant robots, unattendant robots, development robots (similar to unattendant robots but used for development and testing purposes), and non-production robots (similar to attendant robots but used for development and testing purposes). Attendant robots are triggered by user events and operate in parallel with humans on the same computing system. Attendant robots may be used with the conductor 820 for centralized process deployment and logging. Attendant robots may assist human users in accomplishing various tasks and may be triggered by user events. In some embodiments, processes on this type of robot cannot be started from the conductor 820 and / or run under a locked screen. In certain embodiments, the attendant robot can only be started from a robot tray or command prompt. In some embodiments, the attendant robot should be operated under human supervision.

[0111] Unattended robots run unattended in a virtual environment and can automate many processes. Unattended robots can provide remote execution, monitoring, scheduling, and work queue support. In some embodiments, debugging for all robot types can be performed in designer 810. Both attended and unattended robots can automate a wide range of systems and applications, including but not limited to mainframes, web applications, VMs, enterprise applications (e.g., those provided by SAP®, Salesforce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0112] Conductor 820 may have, but is not limited to, various capabilities including provisioning, deployment, configuration, queuing, monitoring, logging, and / or providing interoperability. Provisioning may include creating and maintaining connections between robots 830, AI agents 850, and / or AOP 870 and Conductor 820 (e.g., web applications). Deployment may include ensuring the distribution of the correct package versions for execution to assigned robots 830, AI agents 850, and / or AOPs. Configuration may include maintaining and delivering environment and process configurations for RPA robots and AI agents. Queuing may include providing management of queues and queue items. Monitoring may include tracking identification data for robots and AI agents and maintaining user privileges. Logging may include storing and indexing logs in a database (e.g., a Structured Query Language (SQL) database or a "Not Only" SQL (NoSQL) database) and / or other storage mechanisms (e.g., ElasticSearch®, which provides the ability to store large datasets and query them quickly). Conductor 820 can provide interoperability by acting as a centralized communication point for third-party solutions and / or applications.

[0113] Robot 830 is an execution agent that implements the workflow built in Designer 810. One commercial example of several embodiments of Robot 830 is UiPath Robots®. In some embodiments, the RPA robot 830 installs the Microsoft Windows® Service Control Manager (SCM) management service by default. As a result, such an RPA robot 830 can open an interactive Windows® session under the local system account and has Windows® service privileges.

[0114] In some embodiments, the RPA robot 830 may be installed in user mode. For such a robot 830, this means that it has the same privileges as the user on which it is installed. This functionality may also be available for high-density (HD) robots, which ensure full utilization of the maximum potential of each machine. In some embodiments, any type of RPA robot 830 may be configured in an HD environment.

[0115] In some embodiments, the RPA robot 830 is divided into multiple components, each dedicated to a specific automation task. Robot components in some embodiments may include, but are not limited to, SCM-managed robot services, user-mode robot services, an executor, an agent, and a command line. The SCM-managed robot services manage and monitor Windows® sessions and act as a proxy between the conductor 820 and the execution host (i.e., the computing system on which the robot 830 runs). These services trust and manage the credentials of the RPA robot 830. A console application is launched by the SCM under the local system.

[0116] In some embodiments, the user-mode robot service manages and monitors Windows® sessions and acts as a proxy between the conductor 820 and the execution host. The user-mode robot service may trust and manage the credentials of the RPA robot 830. If the SCM management robot service is not installed, the Windows® application may be launched automatically.

[0117] An executor can execute a given job under a Windows® session (i.e., execute a workflow). An executor can be aware of per-monitor (DPI) settings. An agent can be a Windows® Presentation Foundation (WPF) application that displays available jobs in a system tray window. Note that these agents are different from AI Agent 850. An agent can be a service client and can request to start or stop jobs and change their settings. The command line is also a service client. The command line is a console application that can request to start a job and await its output.

[0118] Dividing the robot 830 into components as described above helps developers, support users, and computing systems more easily execute, identify, and track what each component is doing. In this way, special behavior can be configured for each component, such as setting different firewall rules for the executor and services. In some embodiments, the executor may always be aware of the DPI setting per monitor. As a result, workflows can run at any DPI, regardless of the configuration of the computing system in which they were created. In some embodiments, projects from the designer 810 may also not depend on the browser's zoom level. For DPI-incompatible applications, or applications intentionally marked as incompatible, DPI may be disabled in some embodiments.

[0119] In this embodiment, the agent automation and RPA system 800 is part of a hyperautomation system such as the hyperautomation system 100 in Figure 1. Developers can use the designer 810 to build and test RPA, AOP, and AI agents that utilize AI / ML models deployed in the core hyperautomation system 840 (for example, as part of its AI center). Such RPA robots can send inputs for the execution of AI / ML models and receive outputs from there via the core hyperautomation system 840.

[0120] As described above, one or more of the RPA robots 830 may be listeners. These listeners may provide the core hyperautomation system 840 with information about what the user is doing when using its computing system. This information can then be used by the core hyperautomation system for process mining, task mining, task capture, and the like.

[0121] An assistant / chatbot (not shown) may be provided on the user computing system to enable the user to launch an RPA local robot. For example, the assistant / chatbot may be located in the system tray. The chatbot may have a user interface that allows the user to see the text within the chatbot. Alternatively, the chatbot may not have a user interface and may operate in the background, listening to the user's voice using the computing system's microphone.

[0122] In some embodiments, data labeling may be performed by a user of the computing system on which the RPA robot or AI agent is running, or on another computing system on which the robot or AI agent provides information. For example, if a robot invokes an AI / ML model to perform CV on an image for a VM user, but the AI / ML model fails to correctly identify a button on the screen, the user may draw a rectangle around the misidentified or unidentified component and, if applicable, provide text indicating the correct identification. This information is provided to the core hyperautomation system 540 and can then be used later to train a new version of the AI / ML model.

[0123] Figure 9 is an architecture diagram showing a deployed RPA system 900 according to one embodiment of the present invention. In some embodiments, the RPA system 900 may be part of the agent automation and RPA system 800 in Figure 8, and / or the hyperautomation system 100 in Figure 1. Note that the architecture of the deployed RPA system 900 may not be used in some embodiments. The deployed RPA system 900 may be a cloud-based system, an on-premise system, or a desktop-based system that provides enterprise-level, user-level, or device-level automation solutions for automating different computing processes.

[0124] It should be noted that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, the robot application 910 includes an executor 912, an execution agent 914, and a designer 916. However, in some embodiments, the designer 916 may not be running on the same computing system as the executor 912 and the execution agent 914. The executor 912 is a running process. Multiple business projects may be running simultaneously. In this embodiment, the execution agent 914 (e.g., Windows® service) is a single point of contact for all executors 912. In this embodiment, all messages are recorded in the conductor 940, which further processes them via the database server 950, the AI / ML server 960, the indexer server 970, or any combination thereof. As described above with respect to Figure 8, the executor 912 may be a robot component.

[0125] In some embodiments, an RPA robot represents an association between a machine name and a username. A robot can manage multiple executors simultaneously. On a computing system that supports multiple concurrently running interactive sessions (e.g., Windows® Server 2012), multiple robots can operate simultaneously, each running within a separate Windows® session using a unique username. This is what was referred to above as an HD robot.

[0126] The execution agent 914 is also responsible for transmitting the robot's status (e.g., periodically sending "heartbeat" messages to indicate that the robot is still functioning) and downloading the necessary versions of packages to be executed. In some embodiments, communication between the execution agent 914 and the conductor 940 is always initiated by the execution agent 914. In a notification scenario, the execution agent 914 may open a WebSocket channel which is then used by the conductor 940 to send commands (e.g., start, stop, etc.) to the robot.

[0127] Although not shown here to reduce the complexity of Figure 9, it should be noted that, as described above with respect to Figures 1 and 8, for example, the AI ​​agent can also interact with the conductor 940. The conductor 940 can orchestrate the operation of the AI ​​agent. The conductor 940 can also facilitate interaction between the AI ​​agent and the AI / ML model via the AI / ML server 960, which can store and / or facilitate access to the generated AI model.

[0128] The listener 930 monitors and records data relating to user interaction with the attended computing system in which the listener 930 resides, and / or data relating to the operation of the unattended computing system. The listener 930 may be an RPA robot, part of an operating system, a downloadable application for the computing system, or any other software and / or hardware that does not depart from the scope of the present invention. In fact, in some embodiments, the logic of the listener is partially or completely implemented by physical hardware.

[0129] On the server side, the presentation layer (web application 942, Open Data Protocol (oData) Representative State Transfer (REST) ​​API endpoint 944, and notification and monitoring 946), the service layer (API implementation / business logic 948), and the persistence layer (database server 950, AI / ML server 960, and indexer server 970) are included. Conductor 940 includes the web application 942, the oData REST API endpoint 944, notification and monitoring 946, and the API implementation / business logic 948. In some embodiments, most actions performed by the user on the interface of Conductor 940 (e.g., via a browser 920) are performed by calling various APIs. Such actions may include, but are not limited to, starting jobs on a robot, adding / deleting data in a queue, scheduling jobs to run unattended, etc., without departing the scope of the present invention. The web application 942 is the visual layer of the server platform. In this embodiment, the web application 942 uses HTML and JS. However, any desired markup language, scripting language, or any other form may be used without departing from the scope of the present invention. In this embodiment, the user interacts with web pages from a web application 942 via a browser 920 and performs various actions to control the conductor 940. For example, the user may create robot groups, assign packages to robots, analyze logs per robot and / or per process, and start and stop robots.

[0130] In addition to the web application 942, the conductor 940 also includes a service layer that exposes an oData REST API endpoint 944. However, other endpoints may be included without departing the scope of the present invention. In this embodiment, the REST API is used by both the web application 942 and the execution agent 914. In this embodiment, the execution agent 914 is a supervisor of one or more robots on a client computer.

[0131] The REST API in this embodiment covers configuration, logging, monitoring, and queuing functions. The configuration endpoint may, in some embodiments, be used to define and configure application users, permissions, robots, assets, releases, and environments. The logging REST endpoint may be used to log various information, such as errors, explicit messages sent by robots, and other environment-specific information. The deployment REST endpoint may be used by robots to query the package version to be executed when a start job request is used in conductor 940. The queuing REST endpoint may be responsible for managing queues and queue items, such as adding data to the queue, retrieving transactions from the queue, and setting the state of transactions.

[0132] A monitoring REST endpoint may monitor the web application 942 and the execution agent 914. The notification and monitoring API 946 may be a REST endpoint used for registering the execution agent 914, delivering configuration settings to the execution agent 914, and sending and receiving notifications from the server and the execution agent 914. In some embodiments, the notification and monitoring API 946 may also use WebSocket communication.

[0133] In some embodiments, APIs within the service layer may be accessed through the configuration of appropriate API access paths, for example, based on whether the conductor 940 and the overall hyperautomation system are deployed on-premises or cloud-based. In some embodiments, the API for the conductor 940 may provide custom methods for querying statistics about various entities registered with the conductor 940. Each logical resource may, in some embodiments, be an oData entity. In such entities, components such as robots, processes, and queues may have properties, relationships, and behaviors. In some embodiments, the conductor 940's API may be used by a web application 942 and / or execution agent 914 in two ways: (1) by obtaining API access information from the conductor 940, or (2) by registering an external application and using the oAuth flow.

[0134] The persistence layer in this embodiment includes three servers: a database server 950 (e.g., an SQL server), an AI / ML server 960 (e.g., a server providing AI / ML model serving services such as an AI Center function), and an indexer server 970. The database server 950 in this embodiment stores the configurations of robots and AI agents, groups of robots and AI agents, AOPs, associated processes, users, roles, schedules, etc. This information is managed through a web application 942 in some embodiments. The database server 950 may manage queues and queue entries. In some embodiments, the database server 950 may store messages recorded by robots and AI agents (in addition to, or instead of, the indexer server 970). The database server 950 may also store data related to process mining, task mining, and / or task capture, received, for example, from a listener 930 installed on the client side. Although no arrows are shown between the listener 930 and the database 950, it should be understood that in some embodiments, the listener 930 can communicate with the database 950, and vice versa. This data can be stored in formats such as PDD, images, and XAML files. Note that structured and / or unstructured data may be stored. The listener 930 can be configured to intercept user activity, processes, tasks, and performance metrics on each computing system in which it resides. For example, the listener 930 may record user activity on each computing system (e.g., clicks, input characters, position, application, active element, time, etc.) and then convert them into a format suitable for being provided to and stored by the database server 950.

[0135] The AI / ML Server 960 facilitates the integration of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options make such functionality accessible even to non-data scientists. Deployed automation (e.g., RPA robots and / or AI agents) can invoke AI / ML models from the AI / ML Server 960. AI / ML model performance can be monitored and trained and improved using human-validated data. The AI / ML Server 960 can schedule and execute training jobs to train new versions of AI / ML models. The AI / ML model server can also store and / or access generated AI models.

[0136] The AI / ML server 960 may store data relating to AI / ML models and ML packages for configuring various ML skills for users during development. In this specification, ML skills refer to pre-built and trained ML models for processes, such as those used by automation. The AI / ML server 960 may also store data relating to document understanding techniques and frameworks, as well as algorithms and software packages for various AI / ML capabilities, including but not limited to intent analysis, NLP, speech analysis, and different types of AI / ML models.

[0137] In some embodiments, an optional indexer server 970 stores and indexes information recorded by the robot. In certain embodiments, the indexer server 970 can be disabled through configuration settings. In some embodiments, the indexer server 970 uses ElasticSearch®, an open-source full-text search engine. Messages recorded by the robot (e.g., using activities such as log messages or write lines) are sent to the indexer server 970 via a logging REST endpoint, where they are indexed for future use.

[0138] Figure 10 is an architectural diagram showing the relationships between a designer 1010, activities 1020, 1030, 1040, 1050, a driver 1060, an API 1070, and an AI / ML model 1080 according to one embodiment of the present invention. As described above, the developer uses the designer 1010 to develop workflows and automations to be executed by RPA robots, AI agents, and an AOP engine. The developer can design and configure RPA robot workflows 1012, design and configure agent automation 1014 for AI agents (e.g., providing natural language description, context grounding, tools, etc. for AI agents), and design and configure AOP 1016. See, for example, Figures 4A, 4B, 5, and 6. Various types of activities may be displayed to the developer in some embodiments. The designer 1010 may be local to the user computing system or remote from there (e.g., accessed via a VM or via a local web browser interacting with a remote web server). A workflow for an RPA robot may include a user-defined activity 1020, an API-driven activity 1030, an AI / ML activity 1040, and / or a UI automation activity 1050. The user-defined activity 1020 and the API-driven activity 1040 interact with the application through their respective APIs. In some embodiments, the user-defined activity 1020 and / or the AI / ML activity 1040 may call one or more AI / ML models 1080, which may reside locally in the computing system on which the robot is operating, or remotely from there.

[0139] Some embodiments are capable of identifying non-textual visual elements within an image, which is referred to herein as CV. However, it should be noted that in some embodiments, CV incorporates OCR. CV may be performed at least in part by the AI / ML model 1080. CV activities relating to such elements may include, but are not limited to, text extraction from segmented label data using OCR, fuzzy text matching, clipping of segmented label data using ML, and comparison of extracted text in the label data with ground truth data. In some embodiments, the number of activities that may be implemented in user-defined activity 1020 may number in the hundreds or thousands. However, any number and / or types of activities may be used without departing from the scope of the present invention.

[0140] The UI automation activity 1050 is a subset of special low-level activities written in lower-level code that facilitates interaction with the screen. The UI automation activity 1050 facilitates these interactions via a driver 1060 that enables the robot to interact with desired software. For example, the driver 1060 may include an OS driver 1062, a browser driver 1064, a VM driver 1066, an enterprise application driver 1068, etc. In some embodiments, one or more AI / ML models 1080 may be used by the UI automation activity 1050 to perform interactions with a computing system. In certain embodiments, the AI / ML model 1080 may complement or completely replace the driver 1060. In fact, in certain embodiments, the driver 1060 is not included.

[0141] Driver 1060 can interact with the OS at a low level, such as searching for hooks and monitoring key inputs, via OS driver 1062. Driver 1060 can facilitate integration with Chrome®, IE®, Citrix®, SAP®, and others. For example, a "click" activity can perform the same role in these different applications via driver 1060.

[0142] Figure 11 is an architectural diagram showing a computing system 1100 configured to implement a framework and agent loop for building an AI agent that automatically generates tools and / or uses those tools, according to one embodiment of the present invention. In some embodiments, the computing system 1100 may be one or more computing systems illustrated and / or described herein. In certain embodiments, the computing system 1100 may be part of a hyperautomation system as shown in Figures 1 and 8. The computing system 1100 includes a bus 1105 or other communication mechanism for communicating information and a processor 1110 coupled to the bus 1105 for processing the information. The processor 1110 may be any kind of general-purpose or application processor, including a CPU, ASIC, FPGA, GPU, multiple instances thereof, and / or any combination thereof. The processor 1110 may also have multiple processing cores, at least some of which may be configured to perform specific functions. In some embodiments, multi-parallel processing is used. In certain embodiments, at least one processor 1110 may be a neuromorphic circuit including processing elements that mimic biological neurons. In some embodiments, the neuromorphic circuit may not require the typical components of a von Neumann computational architecture.

[0143] The computing system 1100 further includes memory 1115 for storing information and instructions executed by the processor 1110. Memory 1115 may consist of RAM, ROM, flash memory, cache, static storage devices such as magnetic disks or optical disks, or any other type of non-temporary computer-readable medium, or any combination thereof. The non-temporary computer-readable medium may be any available medium accessible by the processor 1110, and may include volatile media, non-volatile media, or both. The medium may also be removable, non-removable, or both. The computing system 1100 includes a communication device 1120, such as a transceiver, for providing access to a communication network via wireless and / or wired connections. In some embodiments, the communication device 1120 may include one or more antennas, which are single, arrayed, phased, switched, beamforming, beam-steering, combinations thereof, and / or any other antenna configuration that does not depart from the scope of the present invention.

[0144] The processor 1110 is further coupled to the display 1125 via the bus 1105. Any suitable display device and haptic I / O may be used without departing from the scope of the present invention. A keyboard 1130 and a cursor control device 1135, such as a computer mouse or touchpad, are further coupled to the bus 1105 to allow the user to interface with the computing system 1100. However, in certain embodiments, there is no physical keyboard and mouse, and the user can interact with the device only through the display 1125 and / or touchpad (not shown). Any type and combination of input devices may be used as a design choice. In certain embodiments, there is no physical input device and / or display. For example, the user can interact with the computing system 1100 remotely via another computing system communicating with it, or the computing system 1100 may operate autonomously.

[0145] Memory 1115 stores software modules that provide functionality when executed by processor 1110. The modules include an operating system 1140 for the computing system 1100. The modules further include a framework and agent loop implementation module 1145 configured to execute all or part of the processes or derivatives described herein. The computing system 1100 may include one or more additional functionality modules 1150 that provide additional functionality.

[0146] Those skilled in the art will understand that the “computational system” may be implemented as a server, embedded computing system, personal computer, console, personal digital assistant (PDA), mobile phone, tablet computing device, smartwatch, quantum computing system, or any other suitable computing device, or combination thereof, without departing from the scope of the present invention. The fact that the above functions are presented as being performed by a “system” is not intended to limit the scope of the present invention in any sense, but rather to provide an example of one of the many embodiments of the present invention. In fact, the methods, systems, and apparatus disclosed herein may be implemented in local and distributed forms that are consistent with computing technologies, including cloud computing systems. The computing system may be part of, or accessible by, a LAN, mobile communication network, satellite communication network, the Internet, a public cloud, a private cloud, a hybrid cloud, a server farm, or any combination thereof. Any local or distributed architecture may be used without departing from the scope of the present invention.

[0147] It should be noted that some of the system functions described herein are presented as modules to more clearly emphasize their implementation independence. For example, modules may be implemented as hardware circuits including custom very large-scale integrated circuits (VLSI) or gate arrays, commercially available logic chips, transistors, or other individual components. Modules may also be implemented in FPGAs, programmable array logic, programmable logic devices, GPUs, or similar programmable hardware devices.

[0148] Modules can also be implemented, at least partially, as software executed by various types of processors. An identified unit of executable code may include, for example, a block of one or more physical or logical computer instructions that can be configured as an object, procedure, or function. Nevertheless, the executable code of an identified module does not necessarily have to be physically located together, and may include distributed instructions stored in different locations, which together logically constitute the module and achieve its purpose. Furthermore, modules may be stored in a computer-readable medium, which may be, for example, a hard disk drive, flash device, RAM, tape, and / or any other such non-temporary computer-readable medium used to store data without departing from the scope of the present invention.

[0149] In fact, a module of executable code may be a single instruction, a number of instructions, or even distributed across multiple different code segments, different programs, and multiple memory devices. Similarly, operational data may be identified and illustrated within a module herein, embodied in any suitable form, and organized within any suitable type of data structure. Operational data may be collected as a single dataset, or distributed across different locations, including across different storage devices, or at least in part, simply as electronic signals on a system or network.

[0150] Without departing from the scope of the present invention, various types of AI / ML models can be trained and deployed. For example, Figure 12A shows an example of a neural network 1200 trained to complement an agent loop and a framework for building an AI agent that automatically generates tools and / or uses those tools, according to one embodiment of the present invention. The neural network 1200 includes a number of hidden layers. Both DLNNs and shallow learning neural networks (SLNNs) typically have multiple layers, although SLNNs may have only one or two layers in some cases, and usually fewer than DLNNs. Typically, a neural network architecture includes an input layer, multiple hidden layers, and an output layer, as is the case with the neural network 1200.

[0151] DLNNs often have a large number of layers (e.g., 10, 50, 200, etc.), and subsequent layers typically reuse features from previous layers to compute more complex and general functions. SLNNs, on the other hand, tend to have fewer layers and are trained relatively quickly because specialized features are pre-generated from raw data samples. However, feature extraction is time-consuming. In contrast, DLNNs typically do not require specialized features, but training takes longer and they tend to have more layers.

[0152] In both approaches, each layer is trained concurrently on the training set, and overfitting is typically checked on separate cross-validation sets. Both techniques can yield excellent results, and there is considerable expectation for both approaches. The optimal size, shape, and number of individual layers vary depending on the problem each neural network addresses.

[0153] Returning to Figure 12A, the existing agent loop, execution trajectory, context grounding information, APIs for client-side and server-side applications usable by the AI ​​agent, natural language requests, etc., are provided as the input layer and supplied as input to J neurons in Hidden Layer 1. Various other inputs are possible, including, but are not limited to, computational system state information, exposed automation, business rules, information on which RPA workflows and / or tasks are relevant, initial definitions of automation, process automation documents, etc. In this example, all of these inputs are supplied to each neuron, but without departing from the scope of the present invention, feedforward networks, radial basis networks, deep feedforward networks, deep convolutional inverse graphics networks, convolutional neural networks, recurrent neural networks, artificial neural networks, long short-term memory networks, gated recurrent unit networks, generative adversarial networks, liquid state machines, autoencoders, variational autoencoders, denoising autoencoders, sparse autoencoders, extreme learning machines, echo state networks, Markov chains, Hopfield networks, Boltzmann machines, restricted Boltzmann machines, deep residual networks, Kohonen networks, deep belief networks, deep convolutional networks, support vector machines, neural Turing machines, or any other suitable type or combination of neural networks can be used alone or in combination.

[0154] Hidden layer 2 receives input from hidden layer 1, hidden layer 3 receives input from hidden layer 2, and so on, until the last hidden layer provides its output as input to the output layer. Although multiple proposals are shown as outputs here, in some embodiments only a single output proposal is provided. In certain embodiments, proposals are ranked based on confidence scores. In this embodiment, the outputs are possible execution trajectories, execution trajectory steps, execution trajectory paths, and their respective confidence scores.

[0155] It should be noted that the number of neurons I, J, K, and L do not necessarily have to be equal. Therefore, any desired number of neurons can be used in any layer of the neural network 1200 without departing from the scope of the present invention. In fact, in certain embodiments, it is not necessary for all neurons in a given layer to be of the same type.

[0156] The neural network 1200 is trained to assign confidence scores to appropriate outputs. To reduce inaccurate predictions, in some embodiments, only results whose confidence scores meet or exceed a confidence threshold may be provided. For example, if the confidence threshold is 80%, outputs with confidence scores exceeding this value may be used, and the rest may be ignored.

[0157] Neural networks are typically probabilistic constructs that have confidence scores. These can be scores learned by an AI / ML model based on how correctly it identified similar inputs during training. Common types of confidence scores include decimals from 0 to 1 (which can also be interpreted as confidence percentages), numerical values ​​from negative infinity to positive infinity, and a set of representations (e.g., "low," "medium," "high"). Various post-processing calibration techniques such as temperature scaling, batch normalization, weight decay, and negative log-likelihood (NLL) may also be used to obtain more accurate confidence scores.

[0158] In a neural network, "neurons" are typically implemented algorithmically as mathematical functions based on the functions of biological neurons. A neuron receives weighted inputs and has a summation function and activation function that control whether or not to pass the output to the next layer. This activation function can be a nonlinear threshold activity function (i.e., Rectified Linear Unit (ReLU) nonlinearity) that does nothing if the value is below a threshold, but responds linearly when the value exceeds the threshold. The summation function and ReLU function are used in deep learning because real neurons can generally have similar activity functions. Information can be subtracted, added, etc., through linear transformations. Essentially, a neuron functions as a gating function that passes the output to the next layer according to its underlying mathematical function. In some embodiments, different functions may be used for at least some of the neurons.

[0159] An example of neuron 1210 is shown in Figure 12B. Inputs from the preceding layer x1, x2, ..., x n Each of these has weights w1, w2, ..., w n This is assigned. Therefore, the total input from preceding neuron 1 is w1x1. These weighted inputs are used in the summation function of neurons corrected by biases such as the following equation.

number

[0160] This sum is compared to the activation function f(x) to determine whether or not a neuron "fires". For example, f(x) can be given by the following equation:

number

[0161] Therefore, the output y of neuron 1210 can be given by the following equation.

number

[0162] In this case, neuron 1210 is a single-layer perceptron. However, any suitable neuron type or combination of neuron types can be used without departing from the scope of the present invention. It should also be noted that in some embodiments, the range of weight values ​​and / or the range of activation function output values ​​may differ.

[0163] A goal, or "reward function," is often used. The reward function guides the exploration of the state space by searching for intermediate transitions and steps that involve both short-term and long-term rewards, aiming to achieve the goal (e.g., finding the most accurate answer to a user query based on relevant metrics). During training, various labeled data are fed into the neural network 1200. Successful classifications strengthen the weights of the input to the neurons, while failed classifications weaken them. Cost functions such as mean squared error (MSE) or gradient descent can be used to penalize slightly incorrect predictions much less than very incorrect predictions. If the performance of the AI / ML model does not improve after a certain number of training iterations, data scientists may modify the reward function to provide corrections for inaccurate predictions, etc.

[0164] Backpropagation is a technique for optimizing synaptic weights in feedforward neural networks. Backpropagation can be used to reveal the inner workings of the neural network's hidden layers to determine how much of the loss each node is responsible for, and then update the weights to minimize the loss by assigning lower weights to nodes with high error rates and higher weights to nodes with low error rates. In other words, backpropagation allows data scientists to iteratively adjust the weights to minimize the difference between the actual output and the desired output.

[0165] The backpropagation algorithm has a mathematical foundation in optimization theory. In supervised learning, training data with known outputs is passed through a neural network, and an error is calculated from the known target outputs by a cost function, which provides the error for backpropagation. The error is calculated at the output, and this error is converted into corrections to the network weights that minimize the error.

[0166] An example of backpropagation in the case of supervised learning is provided below. A column vector input x is processed through a sequence of N nonlinear activation functions f i between each layer i=1,…,N of the network, and the output at a given layer is first multiplied by the synaptic matrix W i and then added with a bias vector b i . The network output o is given by the following expression.

Formula

[0167] In some embodiments, o is compared with a target output t, resulting in an error

Formula

[0168] Optimization in the form of a gradient descent procedure can be used to minimize the error by modifying the synaptic weights W of each layer i . The gradient descent procedure requires calculating the output o given the input x corresponding to the known target output t, and generating the error o−t. This overall error is then propagated backward to provide local errors for weight updating, the calculation of which is similar but not identical to that used in forward propagation. In particular, the backpropagation step typically requires an activation function in the form of p j (n j )=f j ′(n j ), where n j is the network activity at layer j (that is, n j =Wj o j-1 +b j ) and o j =f j (n j ) and the apostrophe ' represents the derivative of the activity function f.

[0169] Weight updates can be calculated using the following formula:

number

[0170] Here, o represents the Hadamard product (i.e., the element-wise product of two vectors), T represents the matrix transpose, o j is f j (W j o j-1 +b j ) represents and o0 = x. Here, the learning rate η is selected according to machine learning considerations. Below, η is associated with the neural Hebb learning mechanism used in the neural implementation. Note that synapses W and b can be joined into one large synaptic matrix by assuming that 1 is appended to the input vector and an additional column representing the b synapse is included in W.

[0171] An AI / ML model can be trained over multiple epochs until it reaches a good level of accuracy (e.g., 97% or higher using an F2 or F4 threshold for detection, and over approximately 2,000 epochs). This level of accuracy may, in some embodiments, be determined using an F1 score, F2 score, F4 score, or other appropriate technique that does not depart from the scope of the invention. After being trained on training data, the AI / ML model can be tested on evaluation datasets it has not encountered before. This helps prevent the AI / ML model from "overfitting," where it performs well on training data but poorly on other data.

[0172] In some embodiments, the level of accuracy that an AI / ML model can achieve may be unknown. Therefore, if the accuracy of the AI / ML model begins to decline when analyzing evaluation data (i.e., the model performs well on the training data but begins to perform poorly on the evaluation data), the AI / ML model may undergo further training on the training data (and / or new training data) for more epochs. In some embodiments, an AI / ML model is deployed only when its accuracy reaches a certain level, or when the accuracy of the trained AI / ML model is better than that of an existing deployed AI / ML model. In certain embodiments, a set of trained AI / ML models may be used to accomplish a task. For example, one AI / ML model may be trained for image recognition, another for text recognition, and yet another for semantic and / or ontological association recognition.

[0173] It should be noted that, in addition to or instead of neural networks, transformer networks such as SentenceTransformers™ may be used in some embodiments. This is a state-of-the-art Python™ framework for sentence, text, and image embedding. Such transformer networks learn the associations of words and phrases that have both high and low scores. This trains the AI / ML model to determine, respectively, what is close to the input and what is not. Transformer networks may also use field lengths and field types, not just word / phrase pairs.

[0174] As described above, NLP models such as word2vec, BERT, GPT-3, ChatGPT, and other LLMs may be used in some embodiments to facilitate semantic comprehension and provide more accurate and human-like responses. Other techniques, such as clustering algorithms, may be used to find similarities between groups of elements. Clustering algorithms may include, but are not limited to, density-based, distribution-based, centroid-based, and hierarchy-based algorithms. K-means clustering algorithms, DBSCAN clustering algorithms, Gaussian mixture model (GMM) algorithms, and iterative reduction and clustering with hierarchy (BIRCH) algorithms may be used. Such techniques can also assist in categorization.

[0175] Figure 13 is an architecture diagram showing a reference architecture 1300 for a generative AI model according to one embodiment of the present invention. This architecture consists of several layers, namely, an API plugin, a prompt library, vector data source ingestion, access processing control, a model training pipeline, an evaluation layer for evaluating hallucinations / telemetry / evaluation, a BYOM embedding layer, and an LLM orchestration layer. There are also search plugins, access control plugins, and API plugins that are integrated into enterprise systems.

[0176] This embodiment has three main flows.

[0177] Data Ingestion and Training Flow: Data is read from multiple data stores, preprocessed, chunked, and trained through an embedding model (e.g., RAG) and training pipeline (i.e., fine-tuning). A vector database stores the chunked document embeddings, enabling better semantic-based and similarity-based data retrieval.

[0178] Prompt extension using data retrieval: When a user query reaches the API layer, a prompt is selected, and then, before the prompt is passed to the LLM layer, a data retrieval is performed through a vector database or API plugin to obtain the appropriate contextual data.

[0179] LLM Inference: Here, a choice is made between using a general-purpose foundational model or a self-hosted foundational model. Fine-tuned models may be used if they are tailored to a specific task or use case. The response is evaluated, including accuracy and other metrics, including hallucinations.

[0180] It should be noted that in some embodiments, generative AI models having multiple "heads" may be used. A head refers to the output layer of the generative AI model. A generative AI model like the generative AI model 172 in Figure 1 typically has a series of layers, and each head often shares the first few layers of the model before branching out and entering its own layers.

[0181] Figure 14 is a flowchart of a process 1400 for training an AI / ML model according to one embodiment of the present invention. In some embodiments, as described above, the AI / ML model may be a generative AI model. In the case of a neural network, the architecture typically includes multiple neuron layers, including an input layer, an output layer, and hidden layers. See, for example, Figures 12A and 12B. The hidden layers, located between these, generate an intermediate representation of the input that is used to process the input data and produce the output. These hidden layers may include various types of neurons, such as convolutional neurons, recurrent neurons, and / or transformer neurons. A generative AI model may also have various layers.

[0182] In some embodiments, the training process begins in 1410 by providing, labeled or unlabeled, existing agent loops, execution trajectories, context grounding information, APIs for client-side and server-side applications usable by the AI ​​agent, and natural language requests. For generative AI models, which are generally pre-trained, the training process may be omitted unless a fine-tuned model is desired, as will be explained in more detail later. Subsequently, the AI / ML model is trained over several epochs in 1420, and the results are reviewed in 1430. While various types of AI / ML models can be used, LLMs and other generative AI models are typically trained (fine-tuned) using a process called "supervised learning," as described above. Supervised learning involves providing the model with a large dataset, which the model uses to learn the relationship between inputs and outputs. During the training process, the model adjusts the weights and biases of neurons in the neural network to minimize the difference between the predicted output and the actual output in the training dataset.

[0183] One aspect of the model in some embodiments is the use of transfer learning. For example, transfer learning can utilize a pre-trained model such as ChatGPT and fine-tune it for a specific task or domain over 1420 steps. This allows the model to leverage knowledge already learned during the pre-training phase and adapt to a specific application throughout the 1420-step training phase.

[0184] The pre-training phase involves training the model on an initial training dataset, which may be more general. During this phase, the model learns relationships within the data. In the fine-tuning phase (performed in some embodiments in addition to, or instead of, the initial training phase, when the pre-trained model is used as the initial foundation for the final model), the pre-trained model is adapted to a specific task or domain by training it on a smaller dataset specific to that task. For example, in some embodiments, the model may focus on a particular type of data source. This can help the model identify data elements within them more accurately than a generative AI model that has only been pre-trained. Fine-tuning allows the model to learn nuances of the source, such as specific vocabulary and syntax, specific graphical properties, or specific data formats, without requiring as much data as would be needed to train the model from scratch. By leveraging the knowledge learned in the pre-training phase, the fine-tuned model can achieve state-of-the-art performance for a particular task with relatively little additional training data.

[0185] In some embodiments, if the AI / ML model does not meet the desired confidence threshold at 1440, the training data is supplemented and / or the reward function is modified to help the AI / ML model better achieve its objective at 1450, and the process returns to step 1420. If the AI / ML model meets the confidence threshold at 14140, it is tested on evaluation data at 1460 to ensure that the AI / ML model has generalized well and has not overfitted to the training data. The evaluation data contains information that the AI / ML model has not previously processed. If the confidence threshold is met on the evaluation data at 1470, the AI / ML model is deployed at 1480. Otherwise, the process returns to step 1450, and the AI / ML model is further trained.

[0186] Figure 15 shows an agent workflow 1500 according to one embodiment of the present invention. When the workflow starts, the AI ​​agent initializes the process memory, task memory, configuration, and applications used within the process. The text for this step could be, for example, "initialize process memory, initialize task memory (which is hidden from the end user), initialize configuration, and initialize applications used within the process." If initialization fails, the AI ​​agent retries, and may retries a certain number of times. If the retries are unsuccessful, the AI ​​agent instance and the applications being used are terminated.

[0187] If initialization is successful, the AI ​​agent enters the agent loop, where its current state is processed based on the current process memory and task memory. The tools used by the AI ​​agent are also invoked. For static tools, the API definition and implementation are provided within the AI ​​agent definition. For call-specific tools, the AI ​​agent uses the schema definition to determine how the tool should be invoked and how to make the API call itself. If such a tool is used and the tool implementation (e.g., through an API call) is not passed as part of the call, the call may fail at this point. For dynamic tools, the tool definition is provided as part of the AI ​​agent call.

[0188] As described above, tool execution is performed directly by an agent loop that is not interrupted within the AI ​​agent request. The AI ​​agent request simply returns a response to the caller to give a final response or update the UI. Also, as described above, the caller-side tool operates in a similar manner to the server-side tool, but delegates tool execution to the caller within the context of a particular request. In certain embodiments, an AI agent may create another AI agent that generates and / or executes the tool.

[0189] In some embodiments, once the AI ​​agent completes the agent loop, the process proceeds to a human-in-the-loop where information can be retrieved outside the agent loop as needed. If further action by the agent is required, the agent loop continues. If not, the AI ​​agent instance and the application being used are terminated.

[0190] If an exception occurs, the AI ​​agent may first attempt to resolve it within the agent loop. If the AI ​​agent cannot resolve the problem on its own, it will seek human-in-the-loop assistance (for example, through an agent handler with which a human interacts). The text for this step could be, for example, "process any questions from the agent to the agent handler." Depending on the nature of the human response, the process may return to the agent loop or the session may end. If the human agent saves the response provided through the agent handler, the AI ​​agent stores this information in agent memory and uses it for self-correction purposes if a similar problem is encountered in the future.

[0191] In some embodiments, the entire agent process may be automatically constructed from prompts, PDDs, or other sources. For example, a designer application and a generative AI model may determine the agent loop from their sources. The generative AI model may determine from a portion of the PDD that there are specific situations in which the AI ​​agent needs assistance. The designer application and the generative AI model may add steps to the agent loop to handle such situations, such as decision-making logic or steps to escalate to a human-in-the-loop.

[0192] Figure 16 is a flowchart illustrating a framework for building an AI agent that uses dynamic tools and a process 1600 for implementing an agent loop according to one embodiment of the present invention. The process begins in 1605 with the initialization of an AI agent configured to automatically generate tools. If the initialization of the AI ​​agent fails in 1610, the initialization is retried once or multiple times in 1615. If the initialization still fails in 1620, the failure is reported to a human in 1625.

[0193] If steps 1610 or 1620 are successful, the agent loop for the AI ​​agent is entered at 1630. In some embodiments, if the AI ​​agent does not execute the tool directly, the initiating AI agent first creates another AI agent that generates the tool at 1635. The tool-generating AI agent (i.e., the "other" AI agent) has its own agent loop and, in some embodiments, can attempt to analyze and correct the generated tool if the initially generated tool does not conform to the requirements of the initiating AI agent or does not properly respect the I / O schema requested by the initiating AI agent (if such a schema was requested).

[0194] Subsequently, the tool is executed at 1640 by an AI agent or a created AI agent (i.e., a tool-generating AI agent). Alternatively, the initiating AI agent invokes the tool-generating AI agent, and then the initiating AI agent attempts to utilize the tool itself within its own agent loop (e.g., execute the tool) (and may also modify the tool agentically using another agent if necessary). In some embodiments, the tool includes an API definition and implementation (e.g., an executable file, other AI agents, RPA, etc.). In certain embodiments, the execution of the tool is performed directly by the agent loop.

[0195] A valid execution path is enforced at 1645. This may be enforced softly during the execution of the agent loop based on a valid execution path within the prompt, or hardly by validating the tool execution within the agent loop. In some embodiments, the response from the tool execution is returned to the caller by the AI ​​agent to provide a final response from the tool execution or to update the UI. In some cases, the execution of the tool may be delegated to the caller within the context of the request.

[0196] In some embodiments, the tool is a static server-side tool defined abstractly within the AI ​​agent definition and cannot be modified for an AI agent invocation. In certain embodiments, the tool is a call-specific server-side tool defined abstractly within the AI ​​agent definition without describing an API. In some embodiments, the API is provided as part of the AI ​​agent invocation and conforms to the AI ​​agent definition. In certain embodiments, the tool conforms to a schema that describes what each parameter does, what the acceptable values ​​are, what the tool itself does, and when the tool should be used. In some embodiments, the root AI agent (i.e., the originating AI agent) may request another agent to generate and, if applicable, execute a tool, but only for call-specific schema-conformed tools, if the tool was not passed by the caller or if automatic tool generation is explicitly required by the root AI agent definition. For purely dynamic tools, the AI ​​agent may generate such tools that it desires but which are neither mentioned nor provided in the invocation.

[0197] The process steps performed in Figures 14 to 16 may be executed by a computer program that encodes processor instructions for performing at least a portion of the processes described in Figures 14 to 16, according to embodiments of the present invention. The computer program may be implemented on a non-temporary computer-readable medium. The computer-readable medium may include, but is not limited to, a hard disk drive, a flash device, RAM, tape, and / or any other such medium or combination of mediums used to store data. The computer program may include encoded instructions for controlling the processor of a computing system (e.g., processor 1110 of computing system 1100 in Figure 11) to implement all or part of the process steps described in Figures 14 to 16, and these instructions may also be stored on a computer-readable medium.

[0198] Computer programs can be implemented in hardware, software, or a hybrid of both. A computer program may consist of interconnected modules designed to transmit information or instructions to a display. A computer program may be configured to run on a general-purpose computer, an ASIC, or any other suitable device.

[0199] It will be readily apparent that the components of various embodiments of the present invention can be arranged and designed in a wide variety of different configurations, as generally described herein and shown in the drawings. Therefore, the detailed description of embodiments of the present invention shown in the accompanying drawings is not intended to limit the claimed scope of the invention, but merely to illustrate selected embodiments of the invention.

[0200] The features, structures, or characteristics of the present invention described throughout this specification can be combined in any suitable manner in one or more embodiments. For example, any reference throughout this specification to “specific embodiments,” “some embodiments,” or similar phrases means that the specific features, structures, or characteristics described in relation to such embodiments are included in at least one embodiment of the present invention. Therefore, the phrases “in specific embodiments,” “some embodiments,” “other embodiments,” or similar phrases appearing throughout this specification do not necessarily all refer to the same group of embodiments, and the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0201] It should be noted that references to features, advantages, or similar terms throughout this specification do not imply that all features and advantages that can be realized by the present invention should, or actually are, included in any single embodiment of the invention. Rather, terms referring to features and advantages should be understood to mean that the specific features, advantages, or characteristics described in relation to that embodiment are included in at least one embodiment of the invention. Accordingly, discussions of features and advantages, and similar terms throughout this specification may, but not necessarily, refer to the same embodiment.

[0202] Furthermore, the features, advantages, and characteristics of the present invention described herein can be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize that the present invention can be implemented without one or more specific features or advantages of a particular embodiment. In other cases, additional features and advantages that are not present in all embodiments of the present invention may be recognized in a particular embodiment.

[0203] Those skilled in the art will readily understand that the present invention discussed above can be implemented in a different sequence of steps and / or with hardware elements of a different configuration than those disclosed. Therefore, although the present invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will be obvious to them while remaining within the spirit and scope of the invention. Accordingly, to determine the boundaries of the invention, one should refer to the appended claims.

Claims

1. One or more non-temporary computer-readable media, storing one or more computer programs, wherein the one or more computer programs are stored in at least one processor. Initializing an artificial intelligence (AI) agent configured to automatically generate tools, In response to the successful initialization of the AI ​​agent, the agent enters the agent loop for the AI ​​agent. The aforementioned tool is to be executed automatically. It is configured to perform the following: The aforementioned tool is a non-temporary, computer-readable medium that includes application programming interface (API) definitions and implementations.

2. In the one or more non-temporary computer-readable media described in claim 1, The implementation of the aforementioned tool may include an executable file, another AI agent, or robotic process automation (RPA) in a non-temporary, computer-readable medium.

3. In the one or more non-temporary computer-readable media described in claim 1, The execution of the tool is performed directly by the agent loop on a non-temporary, computer-readable medium.

4. In the one or more non-temporary computer-readable media described in claim 1, The one or more computer programs further include the at least one processor, The AI ​​agent returns a response to the caller. The system is configured to perform such actions, and the response is a non-temporary computer-readable medium that provides a final response from the execution of the tool or updates the user interface (UI).

5. In the one or more non-temporary computer-readable media described in claim 1, The one or more computer programs further include the at least one processor, The AI ​​agent creates another AI agent that generates and / or executes the tool. A non-temporary computer-readable medium configured to perform the following actions.

6. In the one or more non-temporary computer-readable media described in claim 1, The aforementioned other AI agent generates the tool, The one or more computer programs further include the at least one processor, In response to the fact that the tool does not meet the requirements of the other AI agent and / or that the tool does not meet the input / output schema requested by the AI ​​agent, the other AI agent automatically analyzes and attempts to correct the generated tool. A non-temporary computer-readable medium configured to perform the following actions.

7. In the one or more non-temporary computer-readable media described in claim 1, The aforementioned other AI agent generates the tool, The one or more computer programs further include the at least one processor, The AI ​​agent executes the tool generated by the aforementioned other AI agent. A non-temporary computer-readable medium configured to perform the following actions.

8. In the one or more non-temporary computer-readable media described in claim 1, The tool is a non-temporary, computer-readable medium, which is a static, call-specific, or dynamic caller-side or server-side tool.

9. In the one or more non-temporary computer-readable media described in claim 1, The tool is a non-temporary, computer-readable medium, which is a static server-side tool defined abstractly within the definition of the AI ​​agent and cannot be modified in response to an invocation of the AI ​​agent.

10. In the one or more non-temporary computer-readable media described in claim 1, The tool is a non-transient, computer-readable medium, which is a call-specific server-side tool abstractly defined within the definition of the AI ​​agent without describing an application programming interface (API).

11. In one or more non-temporary computer-readable media according to claim 10, The API is provided as part of an invocation of the AI ​​agent and conforms to the definition of the AI ​​agent, and is a non-temporary computer-readable medium.

12. In the one or more non-temporary computer-readable media described in claim 1, The tool is a non-temporary, computer-readable medium that conforms to a schema describing what each parameter does, what its tolerances are, what the tool itself does, and when the tool should be used.

13. In the one or more non-temporary computer-readable media described in claim 1, The aforementioned tool is a calling tool, The one or more computer programs further include the at least one processor, Delegating the execution of the aforementioned tool to the caller within the context of the request. A non-temporary computer-readable medium configured to perform the following actions.

14. In the one or more non-temporary computer-readable media described in claim 1, The one or more computer programs further include the at least one processor, The execution of the agent loop is softly forced based on the valid execution path within the prompt. A non-temporary computer-readable medium configured to perform the following actions.

15. In the one or more non-temporary computer-readable media described in claim 1, The one or more computer programs further include the at least one processor, By verifying the validity of tool execution within the agent loop, the execution of the agent loop is to be hard enforced. A non-temporary computer-readable medium configured to perform the following actions.

16. A computer implementation method, The computing system enters an agent loop for artificial intelligence (AI) agents, The aforementioned calculation system automatically executes the tool and Includes, A computer implementation method wherein the tool includes an application programming interface (API) definition and implementation, and the tool is a static, call-specific, or dynamic caller-side tool or server-side tool.

17. In the computer implementation method described in claim 16, further, The aforementioned calculation system softly forces the execution of the agent loop based on the valid execution path within the prompt. A computer implementation method, including

18. In the computer implementation method described in claim 16, further, The calculation system verifies the validity of tool execution within the agent loop, thereby hard-forcing the execution of the agent loop. A computer implementation method, including

19. A single or multiple computing system, Memory for storing computer program instructions, At least one processor configured to execute the aforementioned computer program instructions and Includes, The computer program instruction is provided to at least one processor, Initializing an artificial intelligence (AI) agent configured to automatically generate tools, In response to the successful initialization of the AI ​​agent, the agent enters the agent loop for the AI ​​agent. The aforementioned tool will be executed automatically. It is configured to perform the following actions: One or more computing systems, wherein the tool includes an application programming interface (API) definition and an implementation including an executable file, another AI agent, or robotic process automation (RPA), and the tool is a static, call-specific, or dynamic caller-side or server-side tool.

20. In one or more calculation systems according to claim 19, The tool is a static server-side tool defined abstractly within the definition of the AI ​​agent, which cannot be modified in response to an invocation of the AI ​​agent. The tool is either a call-specific server-side tool defined abstractly within the definition of the AI ​​agent without describing the API, or The API is provided as part of an invocation of the AI ​​agent and comprises one or more computing systems that conform to the definition of the AI ​​agent.

21. In one or more calculation systems according to claim 19, The tool is one or more computing systems that conform to a schema describing what each parameter does, what its tolerances are, what the tool itself does, and when the tool should be used.

22. In one or more calculation systems according to claim 19, The computer program instruction further provides to at least one processor: Based on the valid execution path within the prompt, the execution of the agent loop is softly forced, or By verifying the validity of tool execution within the agent loop, the execution of the agent loop is to be hard enforced. One or more computing systems configured to perform the following:

Images