Framework and agent loop for automatically generating tools and / or building artificial intelligence agents that use tools

Abstract

A framework and an agent loop for automatically generating tools and / or building artificial intelligence (AI) agents that use tools are disclosed. The agent loop can be expressed in business process modeling and notation (BPMN) or any other desired workflow language. These tools can be implemented as static, invocation-specific, or dynamic caller-side or server-side tools. The tools can be defined at all, only abstractly, or fully within the AI agent definition. The implementation of these tools can be injected as part of the agent process. The AI agents that implement these tools can also build other AI agents that can utilize caller-side and server-side tools.

Description

Technical Field

[0001] This invention relates generally to software automation, and more particularly to frameworks and agent loops for automatically generating tools and / or constructing artificial intelligence (AI) agents that use those tools. Background Technology

[0002] AI agents have already been adopted, possessing storage, knowledge bases (e.g., policy and context-based repositories, collections of data, documents, etc. provided by organizations for their specific business and needs), generative AI and large language model (LLM) capabilities for natural language communication, and decision-making capabilities based on historical information. For example, users can send natural language queries to the AI ​​agent via prompts, and the AI ​​agent can provide responses based on its knowledge base and LLM. These AI agents may also have human-machine loopback capabilities, where they can contact human users for queries they cannot effectively resolve. However, these AI agents are often independently developed, standalone software applications, and lack effective interoperability with other applications and software functionalities. Therefore, improved and / or alternative approaches may be beneficial. Summary of the Invention

[0003] Certain embodiments of the present invention may provide solutions and / or useful alternatives to problems and needs that have not yet been fully identified, understood, or resolved by current software automation technologies. For example, some embodiments of the present invention relate to frameworks and agent rings for automatically generating tools and / or constructing AI agents that use those tools.

[0004] In one embodiment, one or more non-transitory computer-readable media store one or more computer programs. The one or more computer programs are configured to cause at least one processor to initialize an AI agent configured to automatically generate a tool. The one or more computer programs are also configured to cause at least one processor to enter an agent loop for the AI ​​agent in response to successful initialization of the AI ​​agent. The one or more computer programs are further configured to cause at least one processor to automatically execute the tool. The tool includes an application programming interface (API) definition and implementation.

[0005] In another embodiment, a computer-implemented method includes entering an agent loop for an AI agent from a computing system. The computer-implemented method also includes automatically executing tools by the computing system. These tools include API definitions and implementations. The tools can be static, call-specific, or dynamic, either caller-side or server-side tools.

[0006] In another embodiment, one or more computing systems include a memory storing computer program instructions and at least one processor configured to execute the stored computer program instructions. The computer program instructions are configured to cause the at least one processor to initialize an AI agent configured to automatically generate a tool. The computer program instructions are also configured to cause the at least one processor to enter an agent loop for the AI ​​agent in response to successful initialization of the AI ​​agent. The computer program instructions are further configured to cause the at least one processor to automatically execute the tool. The tool includes an API definition and an implementation including an executable file, another AI agent, or Robotic Process Automation (RPA). The tool is a static, call-specific, or dynamic caller-side or server-side tool. Attached Figure Description

[0007] To facilitate understanding of the advantages of certain embodiments of the invention, a more specific description of the invention briefly described above will be presented with reference to specific embodiments illustrated in the accompanying drawings. Although it should be understood that these drawings depict only exemplary embodiments of the invention and are therefore not to be considered as limiting its scope, the invention will be described and explained with additional specificity and detail using the drawings, in which:

[0008] Figure 1 This is an architectural diagram illustrating a hyperautomation system configured to perform agent automation and orchestration according to an embodiment of the present invention.

[0009] Figure 2 The illustration shows some of the combined capabilities of an artificial intelligence (AI) agent and a robotic process automation (RPA) robot according to an embodiment of the present invention.

[0010] Figure 3 The illustrations depict an AI agent, an RPA robot, an agent orchestration process (AOP), and an application pool according to embodiments of the present invention.

[0011] Figure 4A and Figure 4B The illustration shows an example AI agent service interface according to an embodiment of the present invention.

[0012] Figure 5 The illustration shows an example AOP development interface according to an embodiment of the present invention.

[0013] Figure 6 An example RPA development interface according to an embodiment of the present invention is illustrated.

[0014] Figure 7 The illustrations depict an end-to-end AI agent, an RPA robot, and an AOP development and deployment system according to embodiments of the present invention.

[0015] Figure 8This is an architectural diagram illustrating an agent automation and RPA system according to an embodiment of the present invention.

[0016] Figure 9 This is an architectural diagram illustrating an RPA system deployed according to an embodiment of the present invention.

[0017] Figure 10 This is an architecture diagram illustrating the relationship between the designer, activities, and drivers according to an embodiment of the present invention.

[0018] Figure 11 This is an architectural diagram illustrating a computing system configured to implement a framework and agent ring for automatically generating tools and / or constructing AI agents that use the tools, according to an embodiment of the present invention.

[0019] Figure 12A An example of a neural network according to an embodiment of the present invention is illustrated, which has been trained to enhance the framework and agent loop for automatically generating tools and / or constructing AI agents that use the tools.

[0020] Figure 12B An example of a neuron according to an embodiment of the present invention is illustrated.

[0021] Figure 13 This is an architectural diagram illustrating a reference architecture for a generative AI model according to an embodiment of the present invention.

[0022] Figure 14 This is a flowchart illustrating the process of training (multiple) AI / ML models according to an embodiment of the present invention.

[0023] Figure 15 The illustration shows an AI agent workflow for an AI agent according to an embodiment of the present invention.

[0024] Figure 16 This is a flowchart illustrating a process for implementing a framework and agent loop for automatically generating tools and / or constructing AI agents that use tools, according to an embodiment of the present invention.

[0025] Unless otherwise indicated, similar reference numerals in all figures always denote corresponding features. Detailed Implementation

[0026] Some embodiments relate to frameworks and agent rings for automatically generating tools and / or building AI agents that use those tools. In some embodiments, the agent ring may be represented in Business Process Modeling and Annotation (BPMN) or any other desired workflow language. As used herein, “tool” refers to an application programming interface (API) definition and implementation (e.g., in a programming language and compiled into an executable, in natural language as another AI agent, in Robotic Process Automation (RPA), etc.).

[0027] Tools can be implemented as static, call-specific, or dynamic caller-side or server-side tools. Static server-side tools are fully, concretely, and specifically defined in the AI ​​agent definition, and the specific call to the AI ​​agent cannot be altered. Call-specific server-side tools have an abstract definition provided in the AI ​​agent definition but do not describe the API. In some embodiments, the API definition may be provided as part of the AI ​​agent call, but it should conform to the definition provided in the AI ​​agent definition. The abstract tool definition is complete from a schema perspective. In other words, the abstract definition defines how the tool should be called, not exactly how the tool should be implemented. For example, consider a document summarization tool that takes Portable Document Format (PDF) documents as input and output strings. In this case, the tool can be implemented in many ways to achieve the same goal of the abstract definition. As another example, the tool could take a structure conforming to schema X (e.g., specified as a JavaScript Object Notation (JSON) schema, including descriptions of the fields in the schema, where the actual tool input must conform to this schema) and output schema Y. Then, at runtime, any tool implementation conforming to input schema X and output schema Y can be dynamically appended. Dynamic server-side tools are only provided as part of the AI ​​agent's invocation.

[0028] As used in this article, "invoking" an AI agent refers to starting or triggering an AI agent to perform a specific task or action. In other words, an AI agent is "invoked" to perform its function based on a given set of inputs or parameters. Invoking mechanisms can include user commands, specific events occurring within the system, predefined schedules prompting the AI ​​agent to activate, etc. Inputs to the AI ​​agent can include questions to be answered, decisions to be made, and sets of parameters to consider.

[0029] For all tool types described above, tool execution is directly handled by the agent loop and is not interrupted within the AI ​​agent request. The AI ​​agent request only returns a response to the caller to obtain a final response or update the user interface (UI), but typically does not provide the caller with control over anything that could change the use of the tool. However, in some embodiments, the caller may be allowed to verify that the tool can be invoked at a specific point in time.

[0030] The same categories (i.e., static, call-specific, and dynamic) exist for caller-side (i.e., client-side) tools. The difference lies in that caller-side tools defer execution to the caller within the context of a specific request (e.g., via bidirectional WebSocket communication between servers or multiple Hypertext Transfer Protocol (HTTP) requests with states passed back to the agent loop to be restored or stored on the server side). This is because only the client can invoke such tools (e.g., create a file, take a screenshot, etc.). This is often useful when agent behavior is built on the caller itself. It should be noted that in both caller-side and server-side cases, the definition of a tool can be represented by a schema (e.g., a JSON schema with annotations) describing what each parameter does, what values ​​are allowed, what the tool itself does, and when the tool should be used.

[0031] As described above, tools may not be defined at all, may be defined only abstractly, or may be fully defined within the AI ​​agent definition. Implementations of these tools can be injected as part of the agent process. In some embodiments, AI agents implementing these tools can build other AI agents capable of utilizing both caller-side and server-side tools. For example, the originating AI agent can invoke a tool with its own agent ring to generate an AI agent (i.e., the "other" AI agent), and in some embodiments, if the initially generated tool does not conform to the requirements of the originating AI agent or does not properly adhere to the input / output (I / O) pattern requested by the originating AI agent (if such a pattern is requested), an attempt can be made to analyze and repair the generated tool. Another option is for the originating AI agent to invoke a tool to generate an AI agent, and then the originating AI agent attempts to consume the tool itself within its agent ring (and, if necessary, potentially repair the tool using another agent).

[0032] In some embodiments, strong guardrails can be provided to mitigate illusions and prevent the AI ​​agent from taking undesirable paths. For example, if a user instructs the AI ​​agent to create a catalog in the file system before creating a PDF file of an invoice, this sequence of actions can be implemented throughout the agent's process (i.e., in agent activities and the customized agent loop). Provide the AI ​​agent with an effective execution trajectory. An "execution trajectory" for an AI agent refers to a specific sequence of actions and decisions taken by the AI ​​agent to complete a task. In other words, an execution trajectory essentially maps the path or route followed by the AI ​​agent from receiving input to producing output within its environment to achieve its goal, including any adjustments or adaptations made along the way based on feedback and changing conditions. For example, an effective execution trajectory can be represented as a directed graph of effective operations and visualized using Mermaid. Effective execution trajectories can be softly constrained, for example, by adding prompts to them, or hard-constrained by validating the effectiveness of tool execution within the agent loop.

[0033] For example, to keep an AI agent on a valid execution trajectory, the system can enable the AI ​​agent to fail, retry, and enforce a specific sequence of execution. This is accomplished via an agent loop for hard constraints or a validating AI agent for soft constraints. This is due to the uncertainty of AI agents with strong guardrails. Without such guardrails, the AI ​​agent is unstable.

[0034] Figure 1 This is an architectural diagram illustrating a hyperautomation system 100 configured to perform agent automation and orchestration according to an embodiment of the present invention. As used herein, "hyperautomation" refers to an automation system that integrates components of process automation, agent automation, integration tools, and technologies that enhance the capabilities of automation. Some examples of these components include, but are not limited to, AI agents, AOP, and RPA robots.

[0035] Typically, as used in this paper, an "AI agent" is an AI-enhanced probabilistic automation that acts independently, dynamically, makes decisions, performs actions, and acts adaptively. This is likely due to the AI ​​agent's use of a Large Language Model (LLM). AI models themselves are often probabilistic in nature. "AOP" is automation that allows users to describe entire business processes. AOP can be created using interfaces that allow the creation of business process diagrams described in a BPMN, which is an Extensible Markup Language (XML) description of business processes. See, for example... Figure 5 "RPA robots" are rule-based, deterministic automations that act predictably and make deterministic decisions.

[0036] For example, in some embodiments, RPA can be used at the core of a hyperautomated system, and in others, automation capabilities can be extended using AI / machine learning (ML), process mining, analytics, agent automation, and / or other advanced tools. For instance, as the hyperautomated system learns processes, trains AI / ML models, and employs analytics, more and more knowledge work can be automated, and computing systems within an organization—such as those used by individuals and those operating autonomously—can participate in the hyperautomated process. Some embodiments of hyperautomated systems allow users and organizations to effectively discover, understand, and extend automation.

[0037] In such embodiments, AI agents coexist in series with RPA robots that perform RPA and AOP. As noted herein, AI agents are automations enhanced with AI skills that can act independently and dynamically make decisions, perform actions, and adjust their performance. AI agents can dynamically utilize available tools via these RPA robots to perform document processing (see, for example, U.S. Patent Application Publication No. 2021 / 0097274), user interface (UI) automation (see, for example, U.S. Patent Nos. 10,654,166, 10,990,876, 11,080,548, 11,507,259, 11,733,668, and 11,748,069), semantic copying and pasting between source and target (see, for example, U.S. Patent Nos. 12,124,806 and U.S. Patent Application Publications Nos. 2023 / 0107316, 2023 / 0415338, and 2024 / 0220581), etc. AI agents can dynamically select these tools and execute them in a pipeline manner.

[0038] Generally speaking, agent automation is probabilistic automation performed by one or more AI agents. Agent automation expands an organization's automation potential by focusing not only on individual tasks but also on the entire end-to-end process. AI agent-guided RPA robot teams can enable a single employee to perform many tasks. Through AI agents, agent automation provides managers with the space to guide, doctors with more time to care for patients, developers with the ability to fine-tune their work, engineers with the freedom to innovate, and customers with seamless and personalized experiences.

[0039] In some embodiments, various technical effects, benefits, and advantages can be achieved through agent automation. Agent automation improves memory usage by requiring less storage for the data and increases processor efficiency by reducing the number of calls and actions. Agent automation also potentially provides the ability to process gigabytes, terabytes, terabytes, or more of data that is impossible for humans to process, whether mentally or manually. Agent automation also potentially enables the use of fewer triggers and models through dynamic decision-making. However, in an example scenario, a standalone RPA might require 100 actions, while with agent automation, this can be significantly reduced (e.g., to 15 actions). Contextual basis can also be used to bind AI agents to the context desired for agent automation. Accordingly, contextual basis “constrains” the LLM to the relevant context.

[0040] AI agents can possess agent memories that develop and remember user interactions, feedback, corrections, and solutions (e.g., dynamic user input from human-machine loopback operations). As used herein, a “human-machine loopback” or human-machine loopback operation can include AI agents and RPA robots that work collaboratively with users to receive dynamic direct user input. In some embodiments, the AI ​​agents (multiple) initially rely 100% on human-machine loopbacks, rather than being trained before being introduced into a production environment. As the agent memory processes and grows to handle human responses, the AI ​​agent can become increasingly autonomous, thereby reducing the need for dynamic direct human input and increasing efficiency. This can be accomplished via Retrieval Augmentation (RAG) or Model Fine-tuning (e.g., using Supervised Fine-tuning (SFT) or Low-Rank Adaptive (LoRA)). Thus, as the AI ​​agent learns to handle increasingly more scenarios, it will seek to autonomously and dynamically select and implement solutions using the multiple tools it has mastered. If more efficient solutions are included or derived from it, the AI ​​agent can also learn from the agent memory that it is more efficient. For example, AI agents can periodically process agent memories to analyze patterns and achieve greater autonomy. Once fully trained, agent automation empowers AI agents to plan, work, and make decisions with minimal human supervision.

[0041] As used in this paper, an "agent memory" is a dynamic cache (i.e., storage) system for managing upgrades and tool calls. Through illustrative operations, when an AI agent encounters a problem at runtime, it can prompt or otherwise request multiple interactions or feedback from the user regarding overcoming the problem, store / cachise these interactions or feedback, and learn from them to reduce the need for repetitive human input. Based on one or more technical effects, benefits, and advantages, the agent memory provides enhanced efficiency by storing solutions to common problems and minimizing potentially costly tool calls. The synergistic operation of the AI ​​agent and the agent memory may "change the trend," thus reducing the need for human interaction as the AI ​​agent continues to learn via the agent memory.

[0042] Generally, agent orchestration is implemented by a command application to fully leverage one or more AOPs (Aspect-Oriented Programming) of AI agents and RPA robots. In some embodiments, agent orchestration orchestrates AI agents (e.g., UiPath Agents). TM ), third-party intelligent agents, RPA robots (e.g., UiPath Robots) TM Agent orchestration encompasses AOP (Aspect-Oriented Programming), and the human agent executing the workflow (e.g., if human approval is required). Agent orchestration thus automates, models, and monitors complex business processes from start to finish. It also provides unique capabilities for orchestrating end-to-end agent workflows across RPA bots, AI agents, third-party agents, and humans. Agent orchestration facilitates the successful scaling of agent automation.

[0043] AI agents for agent automation are based on AI models, as described above, enabling them to work independently of humans and achieve automation. AI agents are also goal-oriented, using context to make probabilistic decisions. Furthermore, AI agents are well-suited for specialized tasks requiring high adaptability. AI agents understand how to perform tasks and improve over time. AI agents can use and select various tools to complete tasks, gather context, and take actions (typically through RPA bots used as tools by the AI ​​agent). In some embodiments, AI agents can build workflows and generate automations for RPA bots and / or other AI agents to execute, such as by leveraging UiPath Autopilot™ for developers or another application that helps developers accelerate the creation and testing of automations. For example, an AI agent can leverage a designer application via API to generate another AI agent or RPA workflow, followed by human-machine loopback to resolve any issues in the generated workflow. If correct, the workflow can be deployed. AI agents can also have varying degrees of autonomy, governed by agent orchestration.

[0044] AI agents generate dynamic plans by executing an "agent loop" to achieve the goal of each instruction using provided tools and context. Once a dynamic plan is generated, the AI ​​agent utilizes efficient execution paths for that plan. If the dynamic plan has two or more steps that can be executed in parallel, the AI ​​agent executes these steps in parallel based on available resources. After each step is completed, the AI ​​agent retrieves the output from that step and regenerates the next one or more steps. Thus, the agent loop continues until the goal is achieved. The parallel execution of dynamic plan steps, based on ecosystem tools and context, is a high-level capability of agent orchestration.

[0045] In some embodiments, the AI ​​agent can also replan after each step. In other words, the initial plan is a suggestion. The AI ​​agent is also able to retrospectively analyze advanced scenarios and identify when it is on the wrong path (e.g., by pre-planning potential paths using mind-tree methods).

[0046] As noted in this paper, RPA bots are rule-based, predictable, and make deterministic decisions. RPA bots are highly reliable, efficient, and well-suited for routine tasks. RPA bots, along with AI agents, can utilize human-robot loopback operations for anomaly management. According to some embodiments, AI agents are more flexible, abstract, and self-deterministic than RPA bots and AOP. RPA bots are generally more stable, concrete, and manageable than AI agents and AOP. AOP processes typically fall between the corresponding flexibility / stability, abstraction / concreteness, and self-deterministic / manageable qualities of AI agents and RPA bots.

[0047] As referenced in this article Figure 3 Furthermore, AI agents and RPA robots can potentially discover and use each other as tools to accomplish tasks. AI agents and RPA robots can also access and use various applications (e.g., via application programming interfaces (APIs)). Tools can be manually configured by developers for automation, and / or AI agents and RPA robots can discover and use tools at runtime.

[0048] According to some embodiments, AI agents, AOPs, and RPA robots can work collaboratively with users (e.g., human-machine loopback), enabling them to make faster, more consistent, and more informed decisions. Furthermore, the use of AI agents, AOPs, and RPA robots allows humans to accomplish more tasks because they can take on additional repetitive, trivial, and ad-hoc tasks on a scale impossible for human users to manage. Humans can make necessary decisions when AI agents, AOPs, or RPA robots encounter anomalies. Therefore, humans can be elevated and focused on acting as supervisors, decision-makers, and organizational leaders.

[0049] AI models provide AI agents with the ability to reason, plan, create, and make autonomous decisions. AI models can also be used by RPA robots for task-specific activities such as processing documents or analyzing data. AI models can be enhanced by leveraging enterprise-specific content and context (e.g., a collection of context stores specific to the enterprise), thereby improving the accuracy and results of the AI ​​models. AI models can be applied individually or simultaneously, depending on the complexity of the task. AI model selection can come from the model library of RPA vendors, third-party models, and Bring Your Own Model (BYOM) options (see, for example, U.S. Patent Nos. 11,738,453 and 11,748,479).

[0050] The hyper-automation system 100 includes user computing systems (such as desktop computers 102, tablets 104, and smartphones 106). However, any desired user computing system can be used without departing from the scope of the invention, including but not limited to smartwatches, laptops, servers, Internet of Things (IoT) devices, etc. Furthermore, although in Figure 1 Three user computing systems are shown, but any suitable number of user computing systems can be used without departing from the scope of the invention. For example, in some embodiments, tens, hundreds, thousands, or millions of user computing systems may be used. The user computing systems may be actively used by users or run automatically without much or any user input.

[0051] As disclosed herein, in some embodiments there are three types of automation: (1) agent automation implemented by a corresponding AI agent; (2) RPA implemented by a corresponding RPA robot; and (3) composite automation implemented by a combination of (multiple) AI agents and (multiple) RPA robots to accomplish a more complex overall task. Automation 110, 112, 114 may include, but is not limited to, those automata performed by RPA robots and / or AI agents, whether performed individually or to achieve a larger composite automation. Other processes (such as listeners) may also be implemented. These processes may be independent applications, subprocesses of another application, parts of an operating system, any other suitable software and / or hardware, or any combination thereof, without departing from the scope of the invention. In fact, in some embodiments, the logic of (multiple) processes is implemented partially or entirely via physical hardware.

[0052] Each user computing system 102, 104, 106 has a corresponding automation 110, 112, 114 (such as those implemented by RPA robots, AI agents, etc.) running on it. In some embodiments, automations 110, 112, 114 may be remotely stored (e.g., on server 130 or in database 140 and accessed via network 120) and loaded by RPA robots and / or AI agents to implement automations 110, 112, 114. Database 140 may store structured and / or unstructured data, although the former is typically required by RPA. RPA automations may exist as scripts (e.g., Extensible Markup Language (XML), Extensible Application Markup Language (XAML), etc.) or be compiled into machine-readable code (e.g., as a digital link library). For example, in the case of AI agents, agent automation may be generated based on a plain text description of the desired objective.

[0053] The listener monitors and records data relating to user interactions with the corresponding computing system and / or the operation of the unattended computing system, and transmits the data to the core hyper-automation system 120 via a network (e.g., a local area network (LAN), mobile communication network, satellite communication network, the Internet, any combination thereof, etc.). The data may include, but is not limited to: which buttons were clicked, where the mouse was moved, text entered in fields, one window being minimized and another being opened, applications associated with the windows, etc. In some embodiments, data from the listener may be periodically sent as part of a heartbeat message. In some embodiments, once a predetermined amount of data has been collected, after a predetermined period of time, or both, the data may be sent to the core hyper-automation system 120. One or more servers (such as server 130) receive the data from the listener and store it in a database (such as database 140).

[0054] In the case of automations 110, 112, and 114 being RPA, automations 110, 112, and 114 can execute logic developed in the workflow during design time. A workflow can include a set of steps executed in sequence or some other logical flow, defined herein as "activities". Each activity can include actions such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows can be nested or embedded.

[0055] In some embodiments, long-running workflows for RPA are the main project supporting service orchestration, human-robot loops, and long-running transactions in unattended environments. See, for example, U.S. Patent No. 10,860,905, the entire contents of which are incorporated herein by reference. Human-robot loops come into play when certain processes require human input (e.g., dynamic direct user input) to handle exceptions, approvals, or verifications before proceeding to the next step in the activity. In this scenario, process execution is paused, and the RPA robot is released until the human-robot loop portion of the task is completed.

[0056] Long-running workflows can support workflow segmentation via persistent activities and can be combined with calling procedures and non-user interactive activities to orchestrate human-machine loopback tasks with RPA bot tasks. In some embodiments, multiple or many computing systems can participate in executing the logic of long-running workflows. Long-running workflows can run in sessions to facilitate rapid execution. In some embodiments, long-running workflows can orchestrate background processes that may include activities that perform API calls and run within the long-running workflow session. In some embodiments, these activities may be invoked by calling procedure activities. Procedures with user interactive activities running in user sessions can be invoked by starting work from a director activity (described in more detail later herein). In some embodiments, the user can interact by requesting a task to complete a form in the director. Activities may be included that cause the RPA bot to wait for the form task to be completed and then resume the long-running workflow.

[0057] One or more of automata 110, 112, and 114 communicate with the core hyperautomation system 120. In some embodiments, the core hyperautomation system 120 may run a command application on one or more servers, such as server 130. Although one server 130 is shown for illustrative purposes, multiple or many servers may be employed in a distributed architecture or adjacent to each other without departing from the scope of the invention. For example, one or more servers may be provided for command functionality, AI / ML model services, authentication, management, and / or any other suitable functionality without departing from the scope of the invention. In some embodiments, the core hyperautomation system 120 may include a public cloud architecture, a private cloud architecture, a hybrid cloud architecture, etc., or may be a part of a public cloud architecture, a private cloud architecture, a hybrid cloud architecture, etc. In some embodiments, the core hyperautomation system 120 may host multiple software-based servers on one or more computing systems, such as server 130. In some embodiments, one or more servers of the core hyperautomation system 120 (such as server 130) may be implemented via one or more virtual machines (VMs).

[0058] In some embodiments, one or more of the automations 110, 112, and 114 can invoke one or more AI / ML models 132 deployed on or accessible by the core hyper-automation system 120 and trained to perform various tasks. For example, AI / ML model 132 may include models trained to find various application versions, perform computer vision (CV), perform optical character recognition (OCR), generate user interface (UI) descriptors, provide suggestions for the next activity or sequence of activities in an RPA workflow, perform semantic matching, perform natural language processing (NLP), generate or modify code and / or RPA workflows, etc. The AI / ML model can be trained using labeled data, including but not limited to elements from a data source (e.g., web pages, forms, scanned documents, application interfaces, screens, etc.), previously created RPA workflows, screenshots of various application screens with various versions of their corresponding UI elements, and a library of UI objects. AI / ML model 132 can be trained to achieve a desired confidence threshold without overfitting to a given training dataset. Generally, UI elements, UI descriptors, applications, and application screens can be considered UI objects.

[0059] AI / ML model 132 can be trained for any suitable purpose without departing from the scope of the invention, as will be discussed in more detail below. In some embodiments, two or more AI / ML models in AI / ML model 132 can be linked (e.g., in series, in parallel, or a combination thereof) such that they collectively provide a synergistic output. AI / ML model 132 can perform or assist CV, OCR, document processing and / or understanding, semantic learning and / or analysis, analytical prediction, process discovery, task mining, testing, automated RPA workflow generation, sequence extraction, cluster detection, audio-to-text translation, NLP, semantic matching, any combination thereof, etc. However, any desired number and / or type of AI / ML models can be used without departing from the scope of the invention. For example, using multiple AI / ML models can allow the system to develop a global picture of what is happening on a given computing system. For example, one AI / ML model can perform OCR, another AI / ML model can detect buttons, another AI / ML model can compare sequences, etc. Patterns can be determined individually by AI / ML models or jointly by multiple AI / ML models. In some embodiments, one or more AI / ML models are locally deployed on at least one of the computing systems 102, 104, 106.

[0060] In some embodiments, multiple AI / ML models 132 may be used. For example, each AI / ML model 132 is an algorithm (or model) that runs on data, and the AI / ML model itself may be a deep learning neural network (DLNN) of trained artificial “neurons” trained on training data. In some embodiments, the AI / ML model 132 may have multiple layers performing various functions, such as statistical modeling (e.g., Hidden Markov Model (HMM)) and utilizing deep learning techniques (e.g., Long Short-Term Memory (LSTM) deep learning, encoding of previous hidden states, etc.) to perform the desired functions.

[0061] In some embodiments, the hyper-automation system 100 may provide four main functional groups: (1) discovery; (2) building automation; (3) management; and (4) engagement. For example, in some embodiments, automation (e.g., running on a user's computing system, server, etc.) may be run by an RPA bot, AOP, or AI agent and may provide any of the functionalities described herein. By way of example, an RPA bot may include an attendant bot, an unattended bot, and / or a test bot. An attendant bot works with the user to assist with tasks (e.g., via UiPath Assistant). TM Unattended bots work independently of users and can run in the background, possibly without the user's knowledge. Test bots run test cases according to the application or RPA workflow. In some embodiments, test bots can be run in parallel on multiple computing systems.

[0062] The discovery function can identify and provide automated recommendations for different opportunities to automate business processes. This function can be implemented by one or more servers (such as server 130). In some embodiments, the discovery function may include providing an automation hub, process mining, task mining, and / or task capture. An automation hub (e.g., UiPath Automation Hub) TM This can provide mechanisms for managing the promotion of automation with visibility and control. For example, automation ideas can be crowdfunded from employees via form submissions. Feasibility and ROI calculations for automating these ideas can be provided, documentation for future automation can be collected, and collaboration can be offered to move from automation discovery to faster builds.

[0063] Process mining (e.g., via UiPath Automation Cloud) TM and / or UiPath AI Center TMProcess mining refers to the process of collecting and analyzing data from applications (e.g., Enterprise Resource Planning (ERP) applications, Customer Relationship Management (CRM) applications, email applications, call center applications, etc.) to identify what end-to-end processes exist within an organization and how to effectively automate them, as well as indicating the impact of automation. This data may be collected from user computing systems 102, 104, 106 by, for example, listeners, and processed by servers (such as server 130). In some embodiments, one or more AI / ML models 132 may be employed for this purpose. This information may be exported to an automation hub to accelerate implementation and avoid manual information delivery. The goal of process mining can be to increase business value by automating processes within an organization. Some examples of process mining goals include, but are not limited to, increasing profits, improving customer satisfaction, regulatory and / or contractual compliance, and improving employee efficiency.

[0064] Task mining (e.g., via UiPath Automation Cloud) TM and / or UiPath AI Center TM The system identifies and aggregates workflows (e.g., employee workflows), and then applies AI to reveal patterns and variations in daily tasks, scoring these tasks for ease of automation and potential savings (e.g., time and / or cost savings). One or more AI / ML models 132 can be employed to reveal repetitive task patterns in the data. Repetitive tasks suitable for automation can then be identified. In some embodiments, this information may initially be provided by a listener and analyzed on a server (such as server 130) of the core hyper-automation system 120. Discoveries from task mining (e.g., XAML process data) can be exported to process documents or to designer applications (such as UiPath Studio). TM (This is for) to create and deploy automation more quickly. In some embodiments, task mining may include taking screenshots using user actions (e.g., mouse click locations, keyboard input, application windows and graphical elements that the user is interacting with, timestamps for interactions, etc.), collecting statistics (e.g., execution time, number of actions, text entries, etc.), editing and annotating screenshots, specifying the types of actions to be recorded, etc.

[0065] Task capture (e.g., via UiPath Automation Cloud) TM and / or UiPath AI Center TMThis automatically records supervised processes or provides a framework for unsupervised processes as users work. This recording can include the expected tasks to be automated in the form of a Process Definition Document (PDD), a skeleton workflow, capturing actions for each part of the process, recording user actions, and automatically generating comprehensive workflow diagrams including details about each step, using Microsoft Word. ® Documentation, XAML files, etc. In some embodiments, the build-ready workflow can be exported directly to a designer application (such as UiPath Studio). TM Task capture can streamline the requirements gathering process for both subject matter experts on the interpretation process and members of the Center of Excellence (CoE) providing production-level automation.

[0066] It can be applied via a designer (e.g., UiPath Studio) TM UiPath StudioX TM or UiPathStudio Web TM This allows for the automation of building processes. For example, developers at RPA development facility 150 can use the designer application 154 of computing system 152 to build and test agent automation, RPA, AOP, and / or for various applications and environments such as web, mobile, and SAP. ® This enables composite automation (including virtualized desktops). Developers can also build AOPs. For example, developers can create automations to be executed by RPA bots, AI agents, AOPs, combinations thereof, etc. API integration is available for various applications, technologies, and platforms. Predefined activities, drag-and-drop modeling, and workflow loggers make automation easier with minimal coding. Document understanding capabilities can be provided via drag-and-drop AI skills that invoke one or more AI / ML models for data extraction and interpretation. This automation can handle virtually any document type and format, including tables, checkboxes, signatures, and handwriting. When data is validated or anomalies are handled, this information can be used to retrain the corresponding AI / ML models, improving their accuracy over time.

[0067] Designer application 152 can be designed to invoke one or more models from trained AI / ML models 132 on server 130 and / or generative AI models 172 in a cloud environment via network 120 (e.g., local area network (LAN), mobile communication network, satellite communication network, the Internet, any combination thereof, etc.) to assist in the automated development process. In some embodiments, one or more AI / ML models may be packaged together with designer application 152 or otherwise stored locally on computing system 150.

[0068] In some embodiments, one or more AI / ML models 132, including designer application 152 and AI / ML models 132, may be configured to use an object store stored in database 140. See, for example, U.S. Patent No. 11,748,069, which is incorporated herein by reference in its entirety. Generally, an object store is a storage mechanism used for automation of images, text, semantic data, classification associations, ontology associations, UI objects, etc. For example, an object store may include a library of UI objects that can be used to develop RPA workflows via designer application 152. The object store can be used for activities that add UI descriptors to the workflow of designer application 152 for UI automation. In some embodiments, one or more AI / ML models 132 may generate new UI descriptors and add them to the object store in database 140.

[0069] Once automation is complete in the designer application 152, it can be published on server 130 and deployed to computing systems 102, 104, 106, etc. For example, when creating new UI descriptors and / or modifying existing ones, a global repository of UI objects can be built for all automations to share and collaborate on. Regarding the object repository, categories and ontology can be used. A category is a hierarchical structure of subclasses. An ontology is a formal representation of a knowledge domain, including concepts, attributes, and the relationships between them. In an ontology, the relationships between categories are not necessarily hierarchical, and ontology relationships can span multiple screens of the application.

[0070] For example, integration services can allow developers to seamlessly combine UI automation with API automation. Automation can be built that requires APIs or traverses both API and non-API applications, as well as systems (such as any of the types described in this article). Repositories (e.g., UiPath ObjectRepository) can be provided for pre-built automation templates and solutions. TM ) or marketplaces (e.g., UiPath Marketplace) TM This allows developers to automate a wide variety of processes more quickly. Therefore, when building automation, the hyperautomation system 100 can provide a user interface, development environment, API integration, pre-built and / or custom-built AI / ML models, development templates, an integrated development environment (IDE), and advanced AI capabilities. In some embodiments, the hyperautomation system 100 enables the development, deployment, management, configuration, monitoring, debugging, and maintenance of RPA robots and AI agents, which can provide automation for the hyperautomation system 100.

[0071] In some embodiments, components of the hyperautomation system 100 (such as (multiple) designer applications and / or external rule engines) provide support for managing and implementing management policies for controlling the various functions provided by the hyperautomation system 100. Management is the ability of an organization to place policies in place to prevent users from developing automation (e.g., RPA bots and / or AI agents) that could take actions that could harm the organization, such as violating the EU General Data Protection Regulation (GDPR), the US Health Insurance Portability and Accountability Act (HIPAA), third-party application terms of service, etc. Because developers can create automation that violates privacy laws, terms of service, etc., while performing their automation, some embodiments implement access control and management restrictions at the bot and / or bot design application level. In some embodiments, this can provide an additional level of security and compliance to the automation process development pipeline by preventing developers from relying on unapproved software libraries that may introduce security risks or work in a manner that violates policies, regulations, privacy laws, and / or privacy policies. See, for example, U.S. Patent No. 11,733,668, which is incorporated herein by reference in its entirety.

[0072] Management functions can provide automated management, deployment, and optimization across organizations. In some embodiments, management functions may include orchestration, test management, AI capabilities, and / or insights. The management functions of the hyper-automation system 100 can also serve as an integration point with third-party solutions and applications for automation applications and / or RPA robots. The management capabilities of the hyper-automation system 100 may include, but are not limited to, facilitating the provisioning, deployment, configuration, queuing, monitoring, logging, and interconnectivity of RPA robots and / or AI agents.

[0073] Commander applications (such as UiPath Orchestrator) TM (It can be provided as UiPath Automation Cloud in some embodiments) TM Part of it, or in the residential area, in a VM, in a private or public cloud, in Linux TM In the VM, or as a result of UiPath Automation Suite TM A cloud-native single container suite provides orchestration capabilities to deploy, monitor, optimize, scale, and securely deploy RPA bots and / or AI agents. Test suites (e.g., UiPath Test Suite) TM It can provide test management to monitor the quality of deployed automation. Test suites can facilitate test planning and execution, requirement fulfillment, and defect traceability. Test suites can include comprehensive test reports.

[0074] Analysis software (e.g., UiPath Insights) TM This software can track, measure, and manage the performance of deployed automation. It can align automated operations with specific key performance indicators (KPIs) and strategic outcomes for the organization. The results can be presented in dashboard formats for better understanding by human users.

[0075] For example, data services (e.g., UiPath Data Service) TM Data can be stored in database 140 and brought to a single, scalable, secure location using a drag-and-drop storage interface. Some embodiments can provide low-code or no-code data modeling and storage for automation while ensuring seamless data access, enterprise-grade security, and scalability. AI capabilities can be provided by an AI center (e.g., UiPath AI Center). TM This system facilitates the integration of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options make this functionality accessible even to those who are not data scientists. Deployed automation (e.g., RPA bots) can invoke AI / ML models (such as AI / ML model 132) from the AI ​​hub. The performance of AI / ML models can be monitored, trained, and improved using human-verified data (such as that provided by the data review center 160). Human reviewers can provide labeled data to the core hyper-automation system 120 via review application 152 on computing system 154. For example, human reviewers can verify that the predictions of AI / ML model 132 and / or generative AI model 172 are accurate or otherwise provide corrections. Human reviewers can also provide dynamic direct user input to the AI ​​agent (e.g., within the scope of human-machine loopback operations), and the responses and corrections provided by human reviewers can be used to train (multiple) LLMs used by the AI ​​agent to be more accurate. For example, in other words, this dynamic input can be saved as training data for retraining AI / ML model 132 and / or generative AI model 172, and can be stored in a database (such as database 140). The AI ​​center can then schedule and execute training tasks to train a new version of the AI / ML model using the training data. Both positive and negative samples can be stored for retraining AI / ML model 132 and / or generative AI model 172.

[0076] Engagement features enable seamless collaboration between people and automation as a team, allowing for the participation of individuals and automation in desired processes. Low-code applications can be built (e.g., via UiPath Apps). TMThis allows for the connection of browser tabs and legacy software, even if some implementations lack an API. For example, applications can be quickly created using a web browser with a rich library of drag-and-drop controls. Applications can be connected to a single automation or multiple automations.

[0077] Action Center (e.g., UiPath Action Center) TM It provides a direct and effective mechanism for switching processes from automation to manual processes, and vice versa. Humans can provide approvals or escalations, make exceptions, etc. Automation can then execute the automatic functions of a given workflow.

[0078] A local assistant can be provided as a launchpad for user-initiated automation (e.g., UiPathAutopilot). TM This assistant can also provide semantic clipping functionality (e.g., UiPath Clipboard AI). TM See, for example, U.S. Patent No. 12,124,806 and U.S. Patent Application Publications Nos. 2023 / 0107316, 2023 / 0415338, and 2024 / 0220581. This functionality can be provided, for example, in a system tray provided by the operating system, and can allow users to interact with RPA robots and RPA robot-powered applications on their computing system. The interface can list automata approved for a given user and allow the user to run them. These can include ready-to-use automata from automation marketplaces, internal automation storage in automation hubs, etc. When automata are running, they can run as local instances in parallel with other processes on the computing system, allowing the user to utilize the computing system while the automation performs its actions. In some embodiments, the assistant is integrated with a task capture function, allowing the user to record the process they are about to automate from the assistant launcher.

[0079] In some embodiments, the automation system 100 can provide end-to-end measurement and management of automation programs of any scale. As described above, analytics can be employed to understand the performance of the automation (e.g., via UiPath Insights). TM Data modeling and analysis using any combination of available business metrics and operational insights can be applied to a wide range of automation processes. Custom-designed and pre-built dashboards allow visualization of data across desired metrics, new analytical insights to be discovered, performance metrics to be tracked, ROI to be discovered for automation, telemetry monitoring to be performed on user computing systems, errors and anomalies to be detected, and automation to be debugged. Automation management consoles (e.g., UiPath Automation Ops) can be provided. TMThis allows organizations to manage automation throughout its entire lifecycle. Organizations can manage how automation is built, what users can do, and which automations users can access.

[0080] In some embodiments, the hyper-automation system 100 provides an iterative platform. It allows for process discovery, the establishment, testing, and deployment of automation, performance measurement, easy provision of automation to users, feedback acquisition, training and retraining of AI / ML models, and the repetition of the process itself. This facilitates more robust and effective automation suites.

[0081] In some embodiments, as described above, generative AI models are used. For example, the AI ​​agent fully utilizes generative AI models. Generative AI models can generate various types of content (such as text, images, audio, and synthetic data). Various types of generative AI models can be used, including but not limited to LLM, generative adversarial networks (GANs), diffusion models, stream-based models, variational autoencoders (VAEs), transformers, etc. In the case of LLM, for example, in some embodiments, NLP models (such as word2vec, BERT, GPT-3, ChatGPT, etc.) can be used to facilitate semantic understanding and provide more accurate and human-like responses. These models can be part of AI / ML model 132 hosted on server 130. For example, generative AI models can be trained on a large corpus of textual information to perform semantic understanding, understand the nature of content present on the screen from text, automatically generate code, etc. The AI ​​agent can use such generative AI models. In some embodiments, existing cloud ML service providers (such as OpenAI) can be employed and trained. ® Google ® Amazon ® Microsoft ® IBM ® Nvidia ® Meta ® This functionality is provided by generative AI model 172 provided by (e.g., [the provider]). In a generative AI embodiment where generative AI model 172 is remotely hosted, server 130 can be configured to integrate with a third-party API, allowing server 130 to send requests to (multiple) generative AI models 172 including necessary input information and receive responses as returns (e.g., semantic matching of fields between application versions, on-screen classification of application types in response to natural language queries from users, etc.). This embodiment can provide a more advanced and sophisticated user experience, as well as access to advanced NLP and other ML capabilities provided by these companies.

[0082] In some embodiments, one aspect of generative AI models is the use of transfer learning. In transfer learning, a pre-trained generative AI model (such as an LLM) is fine-tuned for a specific task or domain. This allows the LLM to leverage knowledge learned during its initial training and adapt it to a specific application. In the case of an LLM, the pre-training phase involves training the LLM on a large text corpus, typically consisting of billions of words. During this phase, the LLM learns relationships between words and phrases, enabling it to generate coherent and human-like responses to text-based input. The output of this pre-training phase is an LLM with a high level of understanding of underlying patterns in natural language.

[0083] During the fine-tuning phase, the pre-trained LLM is adapted to a specific task or domain by training it on a smaller, task-specific dataset. For example, in some embodiments, the LLM can be trained to analyze one or more types of data sources to improve its accuracy regarding their content. This data can include, but is not limited to, cue tuning or instruction tuning, where the model is specifically trained to better understand and follow certain types of instructions or cues, improving its ability to perform a specific task when given appropriate instructions. This information can be provided as part of the training data, and the LLM can learn to focus on these areas and more accurately identify the data elements within them. Fine-tuning allows the LLM to learn the nuances of the task or domain (such as specific vocabulary and grammar used in that domain) without requiring as much data as would be necessary to train the LLM from scratch. By leveraging the knowledge learned in the pre-training phase, a fine-tuned LLM can achieve state-of-the-art performance on a specific task with a relatively small amount of training data.

[0084] LLM can utilize vector databases. Vector databases, along with their vector embeddings, index, store, and provide access to structured or unstructured data, such as text, images, time-series data, etc. Data (such as text) can be lexicalized, where individual letters, words, or sequences of words are parsed from text into lexical units. These lexical units are then "embedded" into vector embeddings, which are numerical representations of the data. Vector databases enable LLM to quickly and proportionally find and retrieve similar objects in production environments, which would be impossible through manual processing.

[0085] AI and ML allow unstructured data to be represented digitally without losing its semantics in vector embeddings. A vector embedding is a long list of numbers, each describing a feature of the data object it represents. Similar objects are grouped together in a vector space. In other words, the more similar the objects, the closer their vector embeddings are to each other. Similar objects can be found using vector search, similarity search, or semantic search. The distance between vector embeddings can be calculated using various techniques, including but not limited to squared Euclidean or L2 squared distance, Manhattan or L1 distance, cosine similarity, dot product, Hamming distance, etc. Choosing the same metric used to train AI / ML models can be beneficial.

[0086] Vector indexing can be used to organize vector embeddings, enabling efficient data retrieval. If a large number of data points exist, calculating the distance between a vector embedding and all other vector embeddings in the vector database using the k-nearest neighbor (kNN) algorithm can be computationally expensive, as the required computation increases linearly with the dimensionality and number of data points (O(n)). Using the Approximate Nearest Neighbor (ANN) method to find similar objects is more efficient. The distances between vector embeddings are pre-computed, and similar vectors are organized and stored close to each other (e.g., in clusters or graphs), allowing for faster finding of similar objects. This process is called "vector indexing." ANN algorithms that can be used in some embodiments include, but are not limited to, cluster-based indexing, proximity graph-based indexing, tree-based indexing, hash-based indexing, compression-based indexing, etc.

[0087] Figure 2 The illustration shows some of the capabilities 200 of the combined capabilities of an AI agent 210 and an RPA robot 220 according to an embodiment of the present invention. The AI ​​agent 210 is configured to process natural language instructions and thereby achieve intended goals, execute using dynamic decision-making or dynamic flow control with self-healing capabilities, store information in long-term memory and evaluate its own execution performance, and learn from human-machine loops and its own performance during execution. The RPA robot 220 can be utilized by the AI ​​agent 210 in response to triggers (e.g., from sources such as UiPath Orchestrator). TMThe RPA robot 220 can perform deterministic steps, such as updating a document based on information retrieved from the context, or alternatively, updating a dynamic plan and executing the next step to achieve the objective according to instructions, based on input from the AI ​​agent. It can also perform actions that the RPA robot can take, such as updating its memory, updating plans to achieve its objectives according to instructions, self-evaluating and learning from actions, self-correcting when encountering obstacles, and escalating to human assistance when needed. Furthermore, it can utilize AI models (e.g., CV models, document processing models, speech-to-text models, OCR models, etc.), RPA tools (e.g., tools available in the RPA ecosystem such as full automation, workflows within automation, integration service connector calls for third-party and first-party services, RPA designer application activities, LLM calls, etc.), and perform actions that the RPA robot can take based on input from the AI ​​agent (i.e., using the RPA robot as a tool).

[0088] As discussed above, various technical effects, benefits, and advantages can be achieved through agent automation in some embodiments. Agent automation improves memory usage by requiring less data storage and increases processor efficiency by reducing the number of calls and actions. Agent automation also potentially provides the ability to process gigabytes, terabytes, terabytes, or more of data that is impossible for humans to process, whether mentally or manually. It also potentially enables the use of fewer triggers and models through dynamic decision-making. However, in an example scenario, a standalone RPA might require 100 actions, while with agent automation, this can be significantly reduced (e.g., to 15 actions). Contextuality can also be used to bind AI agents to the context desired for agent automation. This "constrains" the LLM to the relevant context.

[0089] As used in this paper, "contextual basis" refers to a method of improving models (such as LLMs) by integrating enterprise-specific information with pre-trained knowledge to achieve accurate responses to specific or recent queries. In some embodiments, contextual basis uses external data to enhance LLM responses and obtain responses that the LLM is not inherently aware of, answering queries on top of the provided context. For example, because unique industry terminology and complex document structures can pose challenges in ensuring effective retrieval and semantic matching, contextual basis addresses these challenges by providing precise document chunking to ensure that relevant information (e.g., from unique industry terminology and complex document structures) can be passed to the LLM without noise. With additional examples, contextual basis provides enhanced extraction and search techniques for improved LLM responses tailored to different industries and applications (e.g., for unique industry terminology and complex document structures).

[0090] Figure 3 The illustration shows a pool 300 of AOP, AI agents, RPA robots, and applications according to an embodiment of the present invention. The AOP pool 310 includes AOPs 1, 2, ..., that implement business processes. P As mentioned above, AOP can be implemented as a BPMN, which is executed by an AOP execution engine (such as Temporal). ® AOP can utilize AI agents and / or RPA robots to execute parts of the business process.

[0091] The AI ​​agent pool 320 includes AI agents 1, 2, ..., that have been trained to perform various tasks, such as investigating claims, seeking resolutions with human employees, summarizing strategies and technical specifications, etc. I The RPA robot pool 330 includes RPA robots 1, 2, ..., that perform various automation tasks, such as UI automation, semantic matching automation, form filling automation, etc. J Application pool 340 includes applications that can interact with AI agents and / or RPA robots, 1, 2, ... K For example, applications may include CRM applications, invoicing applications, payroll applications, banking applications, web applications, traditional system applications, word processing applications, spreadsheet applications, email applications, etc. AI agents, RPA robots, and applications may reside on a single computing system or on multiple or many computing systems. AOP is typically in the cloud or otherwise on the server side, and in some embodiments may reside on the same computing systems(s) as the command center application 350.

[0092] AOP can trigger or invoke AI agents and RPA robots via the controller application 350. AI agents and RPA robots can also trigger or invoke each other via the controller application. For example, to invoke an RPA robot, an AI agent can make a "Start Work" call in the controller application 350. It should be noted that the RPA robot is deployed as an automation controlled by the controller application 350. AI agents and RPA robots can also trigger or invoke certain applications. For example, using information collected from human-machine loopback actions, AI agents can dynamically learn which RPA robots, other AI agents, and / or applications should be triggered or invoked to perform tasks. For example, an AI agent can learn to trigger an RPA robot via the controller application 350 to fill out and submit web forms. An AI agent can also learn to open Microsoft Excel. ® This includes inputting form information into the appropriate tabs, opening and updating the payroll application, etc. The AI ​​agent can also learn to invoke or trigger emails via the command center application 350 to resolve issues, which, if a problem occurs, will lead the AI ​​agent to a human customer service representative at the bank. In some embodiments, the technical effects, benefits, and advantages can be similar to those described above regarding... Figure 1 and 2 Those that were discussed.

[0093] To enable AI agents and RPA robots to find each other, AI agents can belong to tenants. The designer application can invoke the director to obtain a list of available RPAs. In some embodiments, there are three ways to obtain automation capabilities: (1) the user provides a description of what the automation does while creating a workflow in the designer application; (2) AI agents and ML technologies are used to generate a summary of what a given workflow does; or (3) the developer can describe what the automation does in the designer application. The director application may also have a list of applications available for a given AI agent and RPA robot. In other words, the descriptions of available AI agents, RPA robots, and / or applications originate from or are assigned by the AI ​​agent, ML technology, or user.

[0094] Figure 4A and Figure 4B An example intelligent agent service interface 400 according to an embodiment of the present invention is illustrated. (Reference) Figure 4AThe agent answers questions about the policy document provided within the context. The agent instruction pane 410 includes a natural language description of what the AI ​​agent wants to do, as input by the user. If needed, user prompts 420 allow developers to enter content in the content field 422 in response to the user prompts. The tool dropdown 430 allows developers to select the tools the AI ​​agent will use (such as using an application-specific API, invoking an RPA bot to perform RPA, etc.).

[0095] The context dropdown 440 allows developers to configure contextual basis for AI agents. The context configuration pane 442 allows developers to provide a description via the description field 444 and an ECS index for a specific policy document via the Elastic Common Schema (ECS) index field 446, which in this example contains information about the contract, regulations, and what to do. Developers can also add additional context 450 to further supplement the contextual basis. Manual escalation options can be configured via dropdown 460.

[0096] Query field 470 allows the user to provide a query that the AI ​​agent will respond to. When the user clicks the run button 480, the AI ​​agent runs the query. Go to Figure 4B Then, as the AI ​​agent retrieves and outputs them, the results during the AI ​​agent's execution are shown in the execution pane 490.

[0097] Figure 5 An example AOP development interface 500 according to an embodiment of the present invention is illustrated. The AOP development interface 500 includes an AOP component 510, an AI agent 520, and an RPA 530, which a user can select when developing a business process. These can be selected and dragged onto a canvas 540, where the user can manually develop AOP. In this example, a credit check is performed by retrieving customer data from a database, invoking an AI agent to determine the customer type (e.g., likely to pay, likely to miss payments, frequent transactions, etc.) by analyzing the customer data. This type is then provided to the RPA robot, which considers this information when performing the credit check. Alternatively, the AOP developer can type a description of the business process into a field 550 and click a generate button 560. This text is provided to an LLM (Local Management System) that attempts to understand the requested business process and automatically create an AOP workflow. The AOP developer can then edit the AOP workflow as needed.

[0098] Figure 6An example RPA development interface 600 according to an embodiment of the present invention is illustrated. The RPA development interface 600 includes RPA components 610 that a user can select when developing an RPA workflow. These can be selected and dragged onto a canvas 620. Alternatively, the RPA developer can type a description of the RPA in a field 630 and click a generate button 640. This text is provided to an LLM (Linux Virtual Machine), which attempts to understand the requested business process and automatically create the RPA workflow. The developer can then edit the RPA workflow as needed. It should be noted that in some embodiments, information about the RPA can be provided in a single designer application. Figure 4A , Figure 4B , Figure 5 and Figure 6 The functions shown and described.

[0099] Figure 7 An end-to-end AI agent, RPA robot, and AOP development and deployment system 700 is illustrated according to an embodiment of the present invention. Designer application 710 allows developers to design AOP, AI agent, and RPA workflows. Once these have been tested and validated, they are packaged and published to automation database 720.

[0100] Commander application 730 manages these automations, as well as the deployment of AOPs, AI agents, and RPA robots. When a human user or software process 732 requests an AOP to be run, commander application 730 sends a start-work request to AOP engine 740, which selects from AOPs 742 and initiates the appropriate automation. When AOP 742 is executed, steps may be encountered that are implemented by AI agents 750 or RPA robots 760. When this occurs, AOP engine 740 pauses the AOP workflow execution and sends a request to commander application 730 to send a start-work request to the appropriate AI agent 750 or RPA robot 760 to execute the step.

[0101] When an AI agent is requested, the command application 730 sends a start-of-work request to the appropriate AI agent 750. This request may include natural language text or other information provided to the command application 730 by the AOP engine 740. The AI ​​agent 750 then performs the step by executing LLM 752 to assist in the task. The AI ​​agent 750 then sends task-related information (e.g., requested information, indications of step completion, indications of step failure, etc.) to the command application 730, which provides this information to the AOP engine 740. The AOP engine 740 then resumes its operation.

[0102] When an RPA robot is requested, the controller application 730 sends a start-of-work request to the appropriate RPA robot 760. The RPA robot 760 then executes the requested RPA 762. The RPA robot 760 then sends task-related information (e.g., requested information, indications of step completion, indications of step failure, etc.) to the controller 730, which provides this information to the AOP engine 740. The AOP engine 740 then resumes its operation.

[0103] In some cases, AOP 742, AI agent 750, or RPA 762 may require human intervention. In such cases, AOP engine 740, AI agent 750, or RPA robot 760 contacts the human 770 for the human-machine loop in the automation. After the human completes the task, AOP engine 740, AI agent 750, or RPA robot 760 resumes the automated portion of the process.

[0104] Figure 8 This is an architectural diagram illustrating an agent automation and RPA system 800 according to an embodiment of the present invention. In some embodiments, the agent automation and RPA system 800 is... Figure 1 This is part of the hyper-automation system 100. The agent automation and RPA system 800 includes a designer 810, which allows developers to design automation for AI agents and RPA robots (e.g., workflows, natural language instructions for AI agents, contextual basis, tool configuration, etc.). The designer 810 can provide solutions for application integration and automating third-party applications, managing information technology (IT) tasks, and business IT processes. The designer 810 can facilitate the development of automation projects that are graphical representations of business processes. Simply put, the designer 810 facilitates the development and deployment of automation for RPA robots and AI agents. In some embodiments, the designer 810 can be an application running on a user's desktop, an application running remotely in a VM, a web application, etc.

[0105] As mentioned above, automation projects automate rule-based processes by giving developers control over the order of execution and the relationships between custom sets of steps (i.e., "activities") developed within the workflow. A commercial example of an embodiment of Designer 810 is UiPath Studio. TM Each activity may include an action (such as clicking a button, reading a file, writing to a log panel, etc.). In some embodiments, workflows may be nested or embedded.

[0106] Some types of workflows can include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. Sequences are particularly well-suited for linear processes, enabling the flow from one activity to another without disrupting the workflow. Flowcharts are particularly well-suited for more complex business logic, allowing decisions and connections between activities to be integrated in more diverse ways through multiple branching logic operators. FSMs are particularly well-suited for large workflows. FSMs can use a finite number of states in their execution, triggered by conditions (i.e., transitions) or activities. Global exception handlers are particularly well-suited for determining workflow behavior and debugging processes when execution errors are encountered.

[0107] Once the workflow and / or other configurations of the AI ​​agents are developed in designer 810, the execution of the business process is coordinated by commander 820, which orchestrates one or more robots 830, one or more AI agents 850, and / or executes one or more AOPs 870 of the workflow developed in designer 810. A commercial example of a commander 820 is the UiPath Orchestrator. TM Commander 820 facilitates the creation, monitoring, and deployment of resources within a management environment. Commander 820 can act as an integration point with third-party solutions and applications. As described above, in some embodiments, commander 820 may be... Figure 1 The core of the hyper-automation system 120.

[0108] It should be noted that the RPA robot 830 can operate independently for deterministic processes. The AI ​​agent 850 and AOP 870 can also operate independently (e.g., for non-deterministic processes), or utilize the RPA robot(s) 830 or other AI agents 850 as tools to complete their agent automation portions. The AI ​​agent 850 can drive composite automation utilizing both the RPA robot 830 and the AI ​​agent 850, and vice versa, and the AOP 870 can include such composite automation.

[0109] Commander 820 can manage a fleet of robots 830 and AI agents 850, connecting and executing RPA robots 830 and AI agents 850 from a central point (e.g., a request from an AOP engine implementing AOP). The types of RPA robots 830 that can be managed include, but are not limited to, supervised robots, unsupervised robots, development robots (similar to unsupervised robots but used for development and testing purposes), and non-production robots (similar to supervised robots but used for development and testing purposes). Supervised robots are triggered by user events and operate alongside humans on the same computing system. Supervised robots can be used with Commander 820 for centralized process deployment and logging platforms. Supervised robots can assist human users in performing various tasks and can be triggered by user events. In some embodiments, processes cannot be started from Commander 820 on this type of robot and / or they cannot run under a locked screen. In some embodiments, supervised robots can only be started from a robot tray or from a command prompt. In some embodiments, supervised robots should operate under human supervision.

[0110] Unattended robots operate unattended in a virtual environment and can automate many processes. Unattended robots can be responsible for remote execution, monitoring, scheduling, and providing support for work queues. In some embodiments, debugging for all robot types can be run in Designer 810. Both attended and unattended robots can automate a wide range of systems and applications, including but not limited to mainframes, web applications, VMs, and enterprise applications (e.g., those developed by SAP). ® Salesforce ® Oracle ® Those generated by [etc.] and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computing applications, etc.).

[0111] Commander 820 may have a variety of capabilities, including but not limited to provisioning, deployment, configuration, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between Robot 830, AI Agent 850, and / or AOP 870 and Commander 820 (e.g., a web application). Deployment may include ensuring that package versions are correctly delivered to assigned Robot 830, AI Agent 850, and / or AOP for execution. Configuration may include the maintenance and delivery of RPA robot and AI agent environments and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot and AI agent identification data and maintaining user permissions. Logging may include storing and indexing logs to a database (e.g., a Structured Query Language (SQL) database or a "not just SQL" (No SQL) database) and / or another storage mechanism (e.g., ElasticSearch®, which provides the ability to store and quickly query large datasets). Commander 820 may provide interconnectivity by acting as a central point of communication for third-party solutions and / or applications.

[0112] Robot 830 is an execution agent that implements the workflow built in designer 810. A commercial example of some embodiments of robot 830 is UiPath Robots. TM In some embodiments, the RPA robot 830 is pre-installed with Microsoft Windows. ® Services managed by the Service Control Manager (SCM). As a result, this RPA robot 830 can open interactive Windows under the local system account. ® Session, and has Windows ® Service permissions.

[0113] In some embodiments, the RPA robot 830 can be installed in user mode. For such a robot 830, this means it has the same permissions as the user to whom the given RPA robot 830 is installed. This feature can also be used for high-density (HD) robots, which ensures full utilization of each machine at its maximum potential. In some embodiments, any type of RPA robot 830 can be configured in an HD environment.

[0114] In some embodiments, the RPA robot 830 is divided into several components, each dedicated to a specific automation task. In some embodiments, robot components include, but are not limited to, SCM-managed robot services, user-mode robot services, actuators, agents, and command lines. SCM-managed robot services manage and monitor Windows. ®The service acts as an agent between the commander 820 and the execution host (i.e., the computing system on which the robot 830 is executed). These services are trusted and manage credentials for the RPA robot 830. The console application is launched by the SCM under the local system.

[0115] In some embodiments, user-mode robot service management and monitoring Windows ® The session acts as an intelligent agent between the command center 820 and the execution host. The user-mode robot service can be trusted and manage credentials for the RPA robot 830. If the SCM-managed robot service is not installed, Windows can be started automatically. ® application.

[0116] The actuator can be used in Windows ® A given job runs within a session (i.e., they can execute workflows). The actuator is aware of the dots per inch (DPI) setting. The agent can be a Windows instance that displays available jobs in a system tray window. ® Demonstrates a basic (WPF) application. Note that these agents are different from AI Agent 850. Agents can be clients of services and can request to start or stop work and change settings. The command line is a client of the service. The command line is a console application that can request to start work and wait for its output.

[0117] As explained above, separating the components of robot 830 helps developers support users and makes it easier for the computing system to run, identify, and track what each component is doing. This allows for configuring specific behaviors for each component (such as establishing different firewall rules for executors and services). In some embodiments, the executor can always know the DPI setting for each monitor. As a result, workflows can be executed at any DPI, regardless of the configuration of the computing system on which the workflow is created. In some embodiments, projects from designer 810 can also be independent of browser scaling levels. For applications where the DPI is unknown or intentionally marked as unknown, DPI can be disabled in some embodiments.

[0118] The agent automation and RPA system 800 in this embodiment is a hyper-automation system (such as...) Figure 1 This is part of the hyper-automation system 100. Developers can use the designer 810 to build and test RPA, AOP, and AI agents that utilize AI / ML models deployed in the core hyper-automation system 840 (e.g., as part of its AI hub). This RPA robot can send inputs to execute the AI / ML models and receive outputs from them via the core hyper-automation system 840.

[0119] As described above, one or more of the RPA robots 830 can be listeners. These listeners can provide the core hyperautomation system 840 with information about what the user is doing while using their computing system. This information can then be used by the core hyperautomation system for process mining, task mining, task capture, etc.

[0120] An assistant / chatbot (not shown) can be provided on the user's computing system to allow the user to launch a local RPA bot. For example, the assistant / chatbot could be located in the system tray. The chatbot can have a user interface so that the user can see the text within the chatbot. Alternatively, the chatbot may lack a user interface and run in the background, using the computing system's microphone to listen to the user's voice.

[0121] In some embodiments, data labeling can be performed by the user of the computing system on which the RPA robot or AI agent is executing, or on another computing system on which the robot or AI agent provides information. For example, if the robot invokes an AI / ML model to perform CV on an image of a VM user, but the AI / ML model does not correctly label buttons on the screen, the user can draw rectangles around incorrectly labeled or unlabeled components, potentially providing text with correct labeling. This information can be provided to the core hyper-automation system 540 and subsequently used to train a new version of the AI / ML model.

[0122] Figure 9 This is an architectural diagram illustrating an RPA system 900 deployed according to an embodiment of the present invention. In some embodiments, the RPA system 900 may be... Figure 8 Intelligent agent automation and RPA systems 800 and / or Figure 1 This is part of the hyper-automated system 100. It should be noted that in some embodiments, the architecture of the deployed RPA system 900 may not be used. The deployed RPA system 900 can be a cloud-based system, a local system, a desktop-based system providing enterprise-level, user-level, or device-level automation solutions for automating different computing processes, etc.

[0123] It should be noted that, without departing from the scope of this invention, the client side, server side, or both may include any desired number of computing systems. On the client side, the robot application 910 includes an actuator 912, an executive agent 914, and a designer 916. However, in some embodiments, the designer 916 may not run on the same computing system as the actuator 912 and the executive agent 914. The actuator 912 is running a process. Several commercial projects can run simultaneously. In this embodiment, the executive agent 914 (e.g., Windows...) ®The service is a single contact point for all actuators 912. All messages in this embodiment are logged in a controller 940, which further processes them via a database server 950, an AI / ML server 960, an indexer server 970, or any combination thereof. (See above regarding...) Figure 8 The actuator 912 discussed here may be a robot component.

[0124] In some embodiments, an RPA robot represents an association between a machine name and a username. The robot can manage multiple actuators simultaneously. This is possible on computing systems that support multiple concurrent interactive sessions (e.g., Windows). ® On a Windows Server 2012, multiple bots can run simultaneously, each using a unique username on a separate Windows machine. ® Running in the session. This is referred to above as the HD robot.

[0125] The executing agent 914 is also responsible for sending the robot's status (e.g., periodically sending "heartbeat" messages indicating that the robot is still working) and downloading the necessary versions of the packages to be executed. In some embodiments, communication between the executing agent 914 and the controller 940 is always initiated by the executing agent 914. In a notification scenario, the executing agent 914 may open a WebSocket channel, which is later used by the controller 940 to send commands to the robot (e.g., start, stop, etc.).

[0126] It should be noted that, although not shown here in order to reduce Figure 9 Despite interference, the AI ​​agent can still interact with the Commander 940, for example, as mentioned above. Figure 1 and 8 The commander 940 can orchestrate the operations of the AI ​​agent. The commander 940 can also facilitate interaction between the AI ​​agent and the AI / ML model via the AI / ML server 960, which can store and / or facilitate access to the generative AI model.

[0127] Listener 930 monitors and records data relating to user interactions with a monitored computing system and / or the operation of an unattended computing system on which listener 930 resides. Without departing from the scope of the invention, listener 930 may be an RPA robot, part of an operating system, a downloadable application for a given computing system, or any other software and / or hardware. In fact, in some embodiments, the logic of the listener is implemented partially or entirely via physical hardware.

[0128] On the server side, there are presentation layers (web application 942, Open Data Protocol (oData) representing State Transfer (REST) ​​Application Programming Interface (API) endpoint 944, and notification and monitoring 946), service layers (API implementation / business logic 948), and persistence layers (database server 950, AI / ML server 960, and indexer server 970). The controller 940 includes web application 942, oData REST API endpoint 944, notification and monitoring 946, and API implementation / business logic 948. In some embodiments, most actions performed by the user in the controller 940's interface (e.g., via browser 920) are performed by calling various APIs. Without departing from the scope of the invention, such actions may include, but are not limited to, starting work on a robot, adding / removing data from a queue, scheduling work to run unattended, etc. Web application 942 is the visual layer of the server platform. In this embodiment, web application 942 uses Hypertext Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or other format may be used without departing from the scope of this invention. In this embodiment, the user interacts with a webpage from web application 942 via browser 920 to perform various actions to control the controller 940. For example, the user can create robot groups, assign packages to robots, analyze logs for each robot and / or each process, start and stop robots, etc.

[0129] In addition to the web application 942, the controller 940 also includes a service layer that displays the oData REST API endpoint 944. However, other endpoints may be included without departing from the scope of the invention. The REST API is consumed by the web application 942 and the executive agent 914. In this embodiment, the executive agent 914 is the supervisor of one or more robots on the client.

[0130] The REST API in this embodiment covers configuration, logging, monitoring, and queuing functions. In some embodiments, the configuration endpoint can be used to define and configure application users, licenses, bots, assets, publications, and environments. The logging REST endpoint can be used to log various information, such as errors, explicit messages sent by bots, and other environment-specific information. The deployment REST endpoint can be used by bots to query the package version that should be executed if a start-work request is used in the controller 940. The queuing REST endpoint can be responsible for queue and queue item management, such as adding data to the queue, retrieving transactions from the queue, setting the status of transactions, etc.

[0131] The monitoring REST endpoint can monitor the web application 942 and the executive agent 914. The notification and monitoring API 946 can be a REST endpoint used to register the executive agent 914, deliver configuration settings to the executive agent 914, and send / receive notifications from the server and the executive agent 914. In some embodiments, the notification and monitoring API 946 can also use WebSocket communication.

[0132] In some embodiments, APIs in the service layer can be accessed by configuring appropriate API access paths, for example, based on whether the Commander 940 and the entire hyper-automation system have a local deployment type or a cloud-based deployment type. The API for the Commander 940 can provide customized methods for querying statistics about various entities registered in the Commander 940. In some embodiments, each logical resource can be an oData entity. Within such an entity, components (such as robots, processes, queues, etc.) can have attributes, relationships, and operations. In some embodiments, the API of the Commander 940 can be consumed by the web application 942 and / or the executive agent 914 in two ways: (1) by obtaining API access information from the Commander 940; or (2) by using an oAuth flow through registration with an external application.

[0133] In this embodiment, the persistence layer includes three servers—a database server 950 (e.g., an SQL server), an AI / ML server 960 (e.g., a server providing AI / ML model services, such as AI center functionality), and an indexer server 970. In this embodiment, the database server 950 stores robot and AI agent configurations, robot and AI agent groups, AOPs, associated processes, users, roles, scheduling, etc. In some embodiments, this information is managed via a web application 942. The database server 950 can manage queues and queue items. In some embodiments, the database server 950 can store messages recorded by robots and AI agents (in addition to or in place of the indexer server 970). For example, the database server 950 can also store data related to process mining, task mining, and / or task capture received from a listener 930 installed on the client side. Although no arrow is shown between the listener 930 and the database server 950, it should be understood that in some embodiments, the listener 930 can communicate with the database server 950, and vice versa. This data can be stored in the form of PDDs, images, XAML files, etc. It should be noted that structured and / or unstructured data can be stored. Listener 930 can be configured to intercept user actions, processes, tasks, and performance metrics on the corresponding computing system where listener 930 resides. For example, listener 930 can record user actions (e.g., clicks, typed characters, locations, applications, active elements, times, etc.) on its corresponding computing system, then convert these into a suitable format for provision to database server 950 and storage in database server 950.

[0134] The AI / ML Server 960 facilitates the integration of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options make this functionality accessible even to those who are not data scientists. Automated deployments (e.g., RPA bots and / or AI agents) can invoke AI / ML models from the AI / ML Server 960. The performance of AI / ML models can be monitored, trained, and improved using human-validated data. The AI / ML Server 960 can schedule and execute training jobs to train new versions of AI / ML models. The AI / ML Server can also store and / or access generative AI models.

[0135] The AI / ML server 960 can store data related to AI / ML models and ML packages for configuring various ML skills of users during development. As used herein, ML skills are pre-built and trained ML models for a process that can be used, for example, by automation. The AI / ML server 960 can also store data related to document understanding techniques, frameworks, algorithms, and software packages for various AI / ML capabilities, including but not limited to intent analysis, NLP, speech analysis, different types of AI / ML models, etc.

[0136] In some embodiments, an optional indexer server 970 stores and indexes information recorded by the robot. In some embodiments, the indexer server 970 can be disabled via configuration settings. In some embodiments, the indexer server 970 uses ElasticSearch. ® It is an open-source full-text search engine project. Messages recorded by the bot (e.g., using activities such as log messages or write lines) can be sent to the indexer server 970 via logging (multiple) REST endpoints, where they are indexed for future use.

[0137] Figure 10 This is an architecture diagram illustrating the relationship 1000 between a designer 1010, activities 1020, 1030, 1040, 1050, a driver 1060, an API 1070, and an AI / ML model 1080 according to an embodiment of the present invention. As described above, developers use the designer 1010 to develop workflows and automations executed by RPA robots, AI agents, and an AOP engine. Developers can design and configure RPA robot workflows 1012, design and configure agent automation 1014 for AI agents (e.g., providing natural language descriptions, contextual information, tools, etc. for AI agents), and design and configure AOP 1016. See, for example, [link to relevant documentation]. Figure 4A , Figure 4B , Figure 5 and Figure 6In some embodiments, various types of activities may be displayed to the developer. The designer 1010 may be local or remote to the user's computing system (e.g., accessed via a VM or a local web browser interacting with a remote web server). Workflows for the RPA robot may include user-defined activities 1020, API-driven activities 1030, AI / ML activities 1040, and / or UI automation activities 1050. User-defined activities 1020 and API-driven activities 1040 interact with the application via their APIs. In some embodiments, user-defined activities 1020 and / or AI / ML activities 1040 may invoke one or more AI / ML models 1080, which may be locally located on the computing system on which the robot is operating and / or remotely located on that computing system.

[0138] Some embodiments are capable of representing non-textual visual components in an image, referred to herein as CV. However, it should be noted that in some embodiments, CV incorporates OCR. CV can be performed at least in part by AI / ML models 1080. Some CV activities associated with such components may include, but are not limited to: extracting text from segmented label data using OCR, fuzzy text matching, cropping segmented label data using ML, comparing text extracted from label data with real data, etc. In some embodiments, hundreds or even thousands of activities can be implemented in user-defined activities 1020. However, any number and / or type of activities may be used without departing from the scope of the invention.

[0139] UI automation activities 1050 are a subset of specific low-level activities written in low-level code and facilitating interaction with the screen. UI automation activities 1050 facilitate these interactions via drivers 1060 that allow the robot to interact with desired software. For example, drivers 1060 may include an operating system (OS) driver 1062, a browser driver 1064, a VM driver 1066, an enterprise application driver 1068, etc. In some embodiments, UI automation activities 1050 may use one or more AI / ML models 1080 to perform interactions with the computing system. In some embodiments, AI / ML models 1080 may enhance drivers 1060 or completely replace them. In fact, in some embodiments, drivers 1060 are not included.

[0140] Driver 1060 can interact with the OS at a low level, such as finding hooks and monitoring keys, via OS driver 1062. Driver 1060 can facilitate interaction with Chrome. ® IE ® Citrix ® SAP ®Integration of features such as "click" activity. For example, the "click" activity performs the same role in these different applications via driver 1060.

[0141] Figure 11 This is an architectural diagram illustrating a computing system 1100 configured to implement a framework and agent loop for automatically generating tools and / or constructing AI agents that use those tools, according to embodiments of the present invention. In some embodiments, the computing system 1100 may be one or more computing systems among those depicted and / or described herein. In some embodiments, the computing system 1100 may be a hyperautomated system (such as...) Figure 1 and 8 (As shown in the diagram). The computing system 1100 includes a bus 1105 or other communication mechanism for transmitting information, and multiple processors 1110 coupled to the bus 1105 for processing information. The multiple processors 1110 can be any type of general-purpose or special-purpose processor, including a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances thereof, and / or any combination thereof. The multiple processors 1110 may also have multiple processing cores, and at least some of these cores may be configured to perform specific functions. In some embodiments, multi-parallel processing may be used. In some embodiments, at least one of the processors 1110 may be a neuromorphic circuit that includes processing elements that simulate biological neurons. In some embodiments, the neuromorphic circuit may not require typical components of a von Neumann computing architecture.

[0142] The computing system 1100 also includes a memory 1115 for storing information and instructions to be executed by the processor(s) 1110. The memory 1115 may consist of random access memory (RAM), read-only memory (ROM), flash memory, cache, static storage (such as a disk or optical disk), or any other type of non-transitory computer-readable medium or any combination thereof. The non-transitory computer-readable medium may be any available medium accessible by the processor(s) 1110 and may include volatile media, non-volatile media, or both. The medium may also be removable, non-removable, or both. The computing system 1100 includes a communication device 1120 (such as a transceiver) to provide access to a communication network via a wireless and / or wired connection. In some embodiments, without departing from the scope of the invention, the communication device 1120 may include one or more antennas, which may be single, arrayed, phased, switched, beamformed, beamguided, combinations thereof, and / or any other antenna configuration.

[0143] The processors 1110 are also coupled to the display 1125 via bus 1105. Any suitable display device and haptic I / O can be used without departing from the scope of the invention. A keyboard 1130 and a cursor control device 1135 (such as a computer mouse, touchpad, etc.) are also coupled to bus 1105 to enable a user to interface with the computing system 1100. However, in some embodiments, a physical keyboard and mouse may be absent, and the user may interact with the device solely through the display 1125 and / or a touchpad (not shown). Any type and combination of input devices can be considered as a design choice. In some embodiments, there are no physical input devices and / or a display. For example, a user may interact with the computing system 1100 remotely via another computing system with which it communicates, or the computing system 1100 may operate autonomously.

[0144] Memory 1115 stores software modules that provide functionality when executed by processor(s) 1110. This module includes an operating system 1140 for computing system 1100. The module also includes an architecture and intelligent agent ring implementation module 1145 configured to perform all or part of the processes described herein or their derivatives. Computing system 1100 may include one or more additional functionality modules 1150, including additional features.

[0145] Those skilled in the art will understand that, without departing from the scope of this invention, the "computing system" can be implemented as a server, embedded computing system, personal computer, console, personal digital assistant (PDA), mobile phone, tablet computing device, smartwatch, quantum computing system, or any other suitable computing device or combination of devices. Presenting the functions described above as being performed by a "system" is not intended to limit the scope of the invention in any way, but rather to provide an example of many embodiments of the invention. In fact, the methods, systems, and apparatuses disclosed herein can be implemented in localized and distributed forms consistent with computing technologies, including cloud computing systems. The computing system can be part of or accessible from a LAN, mobile communication network, satellite communication network, the Internet, public or private cloud, hybrid cloud, server farm, any combination thereof, etc. Any localized or distributed architecture can be used without departing from the scope of this invention.

[0146] It should be noted that some of the system features described in this specification have been presented as modules to more specifically emphasize their implementation independence. For example, modules can be implemented as hardware circuits that include custom-designed very large-scale integrated circuits (VLSI) or gate arrays, off-the-shelf semiconductors (such as logic chips, transistors, or other discrete components). Modules can also be implemented in programmable hardware devices such as field-programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, etc.

[0147] Modules can also be implemented, at least in part, as software executed by various types of processors. Identifying units of executable code can, for example, comprise one or more physical or logical blocks of computer instructions, which can be organized, for example, as objects, procedures, or functions. However, the executable programs of the identified modules do not need to be physically located together, but can include different instructions stored in different locations that, when logically combined, include the module and achieve the module's stated purpose. Furthermore, without departing from the scope of the invention, modules can be stored on a computer-readable medium, which can be, for example, a hard disk drive, a flash memory device, RAM, magnetic tape, and / or any other such non-transitory computer-readable medium for storing data.

[0148] In practice, a module of executable code can be a single instruction or many instructions, and can even be distributed across several different code segments, different programs, and across several storage devices. Similarly, operational data can be identified and represented within a module, and can be embodied in any suitable form and organized within any suitable type of data structure. Operational data can be collected as a single dataset, or it can be distributed across different locations including different storage devices, and can exist at least in part as electrical signals on a system or network.

[0149] Various types of AI / ML models can be trained and deployed without departing from the scope of this invention. For example, Figure 12A An example of a neural network 1200 according to an embodiment of the present invention is illustrated. This neural network 1200 has been trained to enhance a framework and agent loop for automatically generating tools and / or constructing AI agents that use those tools. The neural network 1200 includes multiple hidden layers. DLNNs and shallow learning neural networks (SLNNs) typically have multiple layers, although in some cases an SLNN may have only one or two layers, and is generally fewer than a DLNN. Typically, a neural network structure includes an input layer, multiple intermediate layers, and an output layer, as is the case in neural network 1200.

[0150] DLNNs typically have many layers (e.g., 10, 50, 200, etc.), and subsequent layers often reuse features from previous layers to compute more complex general functions. On the other hand, SLNNs tend to have only a few layers and are relatively fast to train because expert features are created in advance from the original data samples. However, feature extraction is laborious. DLNNs, on the other hand, generally do not require expert features but tend to take longer to train and have more layers.

[0151] For both methods, layers are trained simultaneously on the training set, typically checking for overfitting on isolated cross-validation sets. Both techniques can produce excellent results, and both methods enjoy considerable popularity. The optimal size, shape, and number of individual layers vary depending on the problem each neural network addresses.

[0152] Return to Figure 12A The existing intelligent agent loop, execution trajectory, contextual information, APIs for client and server-side applications that can be used by the AI ​​intelligent agent, natural language requests, etc., are provided as input layers and fed as input to J neurons in hidden layer 1. Various other inputs are possible, including but not limited to computational system state information, published automation, business rules, information about what the RPA workflow and / or tasks involve, initial definitions of automation, process automation documents, etc. Without departing from the scope of the invention, although all these inputs are fed to each neuron in this example, various architectures that can be used individually or in combination are possible, including but not limited to feedforward networks, radial basis function networks, deep feedforward networks, deep convolutional inverse graph networks, convolutional neural networks, recurrent neural networks, artificial neural networks, long / short-term memory networks, gated recurrent unit networks, generative adversarial networks, liquid state machines, autoencoders, variational autoencoders, denoising autoencoders, sparse autoencoders, extreme learning machines, echo state networks, Markov chains, Hopfield networks, Boltzmann machines, restricted Boltzmann machines, deep residual networks, Kohonen networks, deep belief networks, deep convolutional networks, support vector machines, neural Turing machines, or any other suitable type of neural network or combination of neural networks.

[0153] Hidden layer 2 receives input from hidden layer 1, hidden layer 3 receives input from hidden layer 2, and so on, until the last hidden layer provides its output as input to the output layer. While multiple suggestions are shown as outputs here, in some embodiments, only a single output suggestion is provided. In some embodiments, the suggestions are ranked based on confidence scores. In this embodiment, the output is a possible execution trajectory, execution trajectory steps, execution trajectory path, and a corresponding confidence score.

[0154] It should be noted that the number of neurons I, J, K, and L need not be equal. Therefore, without departing from the scope of the invention, any desired number of layers can be used for a given layer of the neural network 1200. In fact, in some embodiments, the types of neurons in a given layer may not all be the same.

[0155] The neural network 1200 is trained to assign confidence scores(s) to appropriate outputs. In some embodiments, to reduce inaccurate predictions, only those results with confidence scores that meet or exceed a confidence threshold can be provided. For example, if the confidence threshold is 80%, outputs with confidence scores exceeding this amount can be used, and the rest can be ignored.

[0156] Neural networks are probabilistic structures that typically have (multiple) confidence scores. These scores can be learned by an AI / ML model based on the frequency with which similar inputs are correctly identified during training. Some common types of confidence scores include decimal numbers between 0 and 1 (which can also be interpreted as confidence percentages), numbers between negative and positive infinity, sets of expressions (e.g., "low," "medium," and "high"), etc. Various post-processing calibration techniques (such as temperature scaling, batch normalization, weight decay, negative log-likelihood (NLL), etc.) can be employed to attempt to obtain more accurate confidence scores.

[0157] In neural networks, "neurons" are algorithmically implemented as mathematical functions, typically based on the functions of biological neurons. Neurons receive weighted inputs and have summing functions and activation functions that manage whether they pass their outputs to the next layer. This activation function can be a non-linear threshold activation function, where nothing happens if the value is below the threshold, but then the function responds linearly above the threshold (i.e., corrected linear unit (ReLU) non-linearity). Since real neurons can have approximately similar activation functions, summing and ReLU functions are used in deep learning. Information can be subtracted, added, etc., via linear transformations. Essentially, neurons act as gating functions, passing their outputs to the next layer as managed by their underlying mathematical functions. In some embodiments, at least some neurons may use different functions.

[0158] Examples of neuron 1210 in Figure 12B As shown in the diagram. Input from the previous layer. x 1, x 2,…, x n Assign corresponding weights w 1, w 2,…, w n Therefore, the collective input from preneuron 1 is w 1 x 1. These weighted inputs are used in neuron summation functions modified by biases, such as:

[0159] The sum and the activation function A comparison is made to determine whether a neuron is "fired". For example, this can be given by the following formula:

[0160] Output of neuron 1210 y Therefore, it can be given by the following formula:

[0161] In this case, neuron 1210 is a single-layer perceptron. However, any suitable neuron type or combination of neuron types may be used without departing from the scope of the invention. It should also be noted that in some embodiments, the weights and / or ranges of the activation function and / or (multiple) output values ​​may differ without departing from the scope of the invention.

[0162] A goal or "reward function" is typically employed. The reward function uses short-term and long-term rewards to explore intermediate transitions and steps to guide the search of the state space and attempt to achieve the goal (e.g., finding the most accurate answer to a user query based on associated metrics). During training, various labeled data are fed through the neural network. Successful labels enhance the weights of the neuron inputs, while unsuccessful labels weaken them. Cost functions (such as mean squared error (MSE) or gradient descent) can be used to penalize slightly incorrect predictions, much less than very incorrect ones. If the performance of the AI / ML model does not improve after a certain number of training iterations, data scientists can modify the reward function to provide corrections for incorrect predictions, etc.

[0163] Backpropagation is a technique used to optimize synaptic weights in feedforward neural networks. It can be used to "pop the hood" of the hidden layers of a neural network to see how much loss each node is responsible for, and then update the weights in a way that minimizes the loss by giving lower weights to nodes with higher error rates, and vice versa. In other words, backpropagation allows data scientists to repeatedly adjust the weights to minimize the difference between the actual and expected outputs.

[0164] The backpropagation algorithm is mathematically established in optimization theory. In supervised learning, training data with known outputs is passed through a neural network, and the error is calculated using a cost function derived from the known target output, which gives the error for backpropagation. The error is then calculated at the output and converted into corrections to the network weights that minimize the error.

[0165] In the context of supervised learning, an example of backpropagation is provided below. This involves propagating the network through each layer. A series of N A nonlinear activity function To process column vector input xThe output of a given layer is first multiplied by the synaptic matrix. W i And add a bias vector b i Network output o It is given by the following formula:

[0166] In some embodiments, o With target output t By comparing, we can obtain the error that is expected to be minimized. .

[0167] By modifying the synaptic weights for each layer W i The error can be minimized using an optimization procedure in the form of gradient descent. The gradient descent procedure requires a known target output. t Corresponding input x Calculate the output under the following circumstances o And it produces errors. o - t Then, this global error is backpropagated, giving a local error for the weight update, which has a computation similar to, but not exactly the same as, the computation used for forward propagation. In particular, the backpropagation step typically requires the form... The activity function, where n j It is a layer j Network activities at the location (i.e., ),and Represents the activity function The derivative of .

[0168] The weight update can be calculated using the following formula:

[0169] in This represents the Hadamard product (i.e., the element-wise product of two vectors). T This represents the matrix transpose, and o j express ,in Here, the learning rate is selected based on machine learning considerations. The following text... This relates to the neural Hebbian learning mechanism used in the neural implementation. Note that synapses can be... W and b Combined into a large synaptic matrix, where it is assumed that the input vector has an additional 1, and will represent b The additional column for each synapse is incorporated into W middle.

[0170] The AI / ML model can be trained over multiple epochs until it reaches a good level of accuracy (e.g., 97% or better, using F2 or F4 thresholds for detection, approximately 2000 epochs). In some embodiments, this level of accuracy can be determined using F1 scores, F2 scores, F4 scores, or any other suitable techniques without departing from the scope of the invention. Once trained on the training data, the AI / ML model can be tested on evaluation datasets that have not been encountered before. This helps ensure that the AI / ML model is not "overfitting," meaning it performs well on the training data but poorly on other data.

[0171] In some embodiments, the potential accuracy level of the AI / ML model may be unknown. Accordingly, if the accuracy of the AI / ML model begins to decline when analyzing evaluation data (i.e., the model performs well on training data but begins to perform poorly on evaluation data), the AI / ML model may undergo more training epochs on the training data (and / or new training data). In some embodiments, the AI / ML model is deployed only if a certain level of accuracy is achieved or if the accuracy of the trained AI / ML model is superior to that of existing deployed AI / ML models. In some embodiments, a collection of trained AI / ML models can be used to accomplish the task. For example, one AI / ML model may be trained to recognize images, another AI / ML model to recognize text, yet another AI / ML model to recognize semantics and / or ontology associations, etc.

[0172] It should be noted that, in addition to or instead of neural networks, some embodiments may use transformer networks (such as SentenceTransformers). TM It is a Python library for advanced sentence, text, and image embedding. TM The framework involves a transformer network that learns associations between words and phrases with high and low scores. This trains AI / ML models to determine what is close to the input and what is not. The transformer network can also use field length and field type, not just word / phrase pairs.

[0173] As mentioned above, NLP models (such as word2vec, BERT, GPT-3, ChatGPT, and other LLMs) can be used in some implementations to enhance semantic understanding and provide more accurate and human-like responses. Other techniques (such as clustering algorithms) can be used to discover similarities between groups of elements. Clustering algorithms can include, but are not limited to, density-based algorithms, distribution-based algorithms, centroid-based algorithms, hierarchical algorithms, K-means clustering, DBSCAN clustering, Gaussian mixture model (GMM) algorithms, and Balanced Iterative Reduction and Clustering (BIRCH) algorithms using hierarchical structures. These techniques can also assist in classification.

[0174] Figure 13 This is an architecture diagram illustrating a reference architecture 1300 for a generative AI model according to an embodiment of the present invention. The architecture consists of several layers: an API plugin, a hint library, vector data source ingestion, access processing control, a model training pipeline, an evaluation layer for evaluation illusion / telemetry / evaluation, a BYOM embedding layer, and an LLM orchestration layer. There are also retrieval plugins, access control plugins, and API plugins integrated into enterprise systems.

[0175] There are three main types in this embodiment:

[0176] Data ingestion and training stream This involves reading, preprocessing, chunking, and training data from multiple data stores through embedding models (e.g., Retrieval Augmentation Generation (RAG)) and training pipelines (i.e., fine-tuning). Vector database storage allows for chunked document embeddings for better semantic, similarity-based data retrieval.

[0177] Enhanced prompts for data retrieval Once a user query reaches the API layer, a hint is selected, and data is then retrieved via a vector database or API plugin to obtain the correct contextual data before passing the hint to the LLM layer.

[0178] LLM Reasoning This involves choosing between using a general-purpose base model or a self-hosted base model. A fine-tuned model can be used when tuning for a specific task or use case. Evaluate the accuracy of the response and other metrics, including illusions.

[0179] It should be noted that in some embodiments, generative AI models with multiple "heads" can be used. A head refers to the output layer of the generative AI model. Generative AI models (such as...) Figure 1 Generative AI models (172) typically have a sequence of layers, and each head usually shares the first few layers of the model before being distributed to its own different layers.

[0180] Figure 14This is a flowchart illustrating a process 1400 for training (multiple) AI / ML models according to an embodiment of the present invention. In some embodiments, as described above, the (multiple) AI / ML models may be generative AI models. In the case of neural networks, this architecture typically includes multiple layers of neurons, including input, output, and hidden layers. See, for example, [link to documentation]. Figure 12A and Figure 12B The hidden layers process the input data and generate intermediate representations of the input used to generate the output. These hidden layers can include various types of neurons (such as convolutional neurons, recurrent neurons, and / or transform neurons). Generative AI models can also have various layers.

[0181] At 1410, in some embodiments, the training process begins by providing an existing agent loop, execution trajectory, contextual information, APIs for client and server-side applications that the AI ​​agent can use, natural language requests, etc., whether labeled or unlabeled. In the case of generative AI models that are typically trained, the training process can be skipped unless fine-tuning is desired, as discussed in more detail below. The AI / ML model is then trained over multiple epochs at 1420, and the results are checked at 1430. While various types of AI / ML models can be used, LLM and other generative AI models are typically trained (fine-tuned) using a process also discussed above, known as “supervised learning.” Supervised learning involves providing the model with a large dataset, which is then used to learn the relationship between inputs and outputs. During the training process, the model adjusts the weights and biases of neurons in the neural network to minimize the difference between the predicted outputs and the actual outputs in the training dataset.

[0182] One aspect of the model in some embodiments is the use of transfer learning. For example, transfer learning may leverage a pre-trained model (such as ChatGPT) that is fine-tuned for a specific task or domain in step 1420. This allows the model to utilize knowledge already learned from the pre-training phase and adapt it to a specific application via the training phase of step 1420.

[0183] The pre-training phase involves training the model on a potentially more general initial training dataset. During this phase, the model learns relationships within the data. In the fine-tuning phase (e.g., in some embodiments, performed during step 1420 in addition to or in place of the initial training phase if the pre-trained model is used as the initial basis for the final model), the pre-trained model is adapted to a specific task or domain by training it on a smaller, task-specific dataset. For example, in some embodiments, the model may focus on certain types of data sources. This can help the model identify data elements more accurately than a generative AI model pre-trained alone. Fine-tuning allows the model to learn nuances in the sources (such as specific vocabulary and grammar, certain graphical features, certain data formats, etc.) without requiring as much data as is necessary to train the model. By leveraging the knowledge learned in the pre-training phase, the fine-tuned model can achieve state-of-the-art performance for a specific task with relatively little additional training data.

[0184] If, in some embodiments, the AI / ML model fails to meet the expected confidence threshold at 1440, then at 1450, supplementary training data and / or modification of the reward function are performed to help the AI / ML model better achieve its goal, and the process returns to step 1420. If the AI / ML model meets the confidence threshold at 1440, then at 1460, the AI / ML model is tested on evaluation data to ensure that the AI / ML model generalizes well and is not overfitting relative to the training data. The evaluation data includes information that the AI / ML model has not previously processed. If the evaluation data meets the confidence threshold at 1470, then the AI / ML model is deployed at 1480. If not, the process returns to step 1450, and the AI / ML model is trained further.

[0185] Figure 15 The diagram illustrates an AI agent workflow 1500 according to an embodiment of the present invention. When the workflow begins, the AI ​​agent initializes the process memory, task memory, configuration, and the application to be used in the process. The text for this step might be, for example, "Initialize process memory, initialize task memory (which will be hidden from the end user), initialize configuration, and initialize the application used in the process." If initialization fails, the AI ​​agent retryes and can retry a certain number of times. If the retry is unsuccessful, the used AI agent instance and application are shut down.

[0186] If initialization is successful, the AI ​​agent enters the agent loop, where the current state is processed based on the current process and task memory. Tools used by the AI ​​agent are also invoked. In the case of static tools, the API definition and implementation are provided in the AI ​​agent definition. In the case of invoking a specific tool, the AI ​​agent uses a pattern definition to determine how the tool should be invoked and specifies how to make the API call itself. If such a tool is used and the tool implementation (e.g., via API call) is not passed as part of the call, the call may fail. In the case of dynamic tools, the tool definition is provided as part of the AI ​​agent call.

[0187] As described above, the tool's execution is directly handled by the agent ring, without interruption within the AI ​​agent's request. The AI ​​agent request only returns a response to the caller to provide a final answer or update the UI. Similarly, as mentioned above, the caller-side tool operates in a manner similar to the server-side tool, but its execution is deferred to the caller within the context of a specific request. In some embodiments, the AI ​​agent can create another AI agent that generates and / or executes the tool.

[0188] In some embodiments, if the AI ​​agent completes the agent loop, the process proceeds to the human-machine loop, where the human can retrieve information outside the agent loop if needed. If further action by the agent is required, the agent loop continues. If not, the used AI agent instance and application are shut down.

[0189] If an anomaly occurs, the AI ​​agent can first attempt to resolve the problem within the agent loop. If the AI ​​agent is unable to resolve the problem on its own, it seeks human-machine loop assistance (e.g., via an agent processor with which a human interacts). The text for this step could be, for example, "The agent processes any problem with the agent processor." Depending on the nature of the human response, the process may return to the agent loop, or the session may end. If the AI ​​agent saves the response provided via the agent processor, it stores this information in its agent memory and uses it for self-correction purposes if it encounters a similar problem in the future.

[0190] In some embodiments, the entire agent process can be automatically constructed from prompts, PDDs, or other sources. For example, designer applications and generative AI models can determine the agent loop from the sources. The generative AI model can identify certain scenarios where the AI ​​agent needs assistance from portions of the PDD. Designer applications and generative AI models can add steps to the agent loop to handle these scenarios, such as decision logic and steps escalating to the human-machine loop.

[0191] Figure 16This is a flowchart illustrating a process 1600 for implementing a framework and agent loop for constructing AI agents using dynamic tools, according to an embodiment of the present invention. The process begins at 1605 with the initialization of the AI ​​agent configured as an automatically generated tool. If the initialization of the AI ​​agent fails at 1610, it is retried one or more times at 1615. If the initialization still fails at 1620, the failure is reported to a human at 1625.

[0192] If steps 1610 or 1620 are successful, the AI ​​agent enters its agent loop at 1630. In some embodiments, if the AI ​​agent does not directly execute the tool, the originating AI agent creates another AI agent at 1635 to generate the tool first. The tool-generating AI agent (i.e., the "other" AI agent) has its own agent loop, and in some embodiments, if the initially generated tool does not conform to the requirements of the originating AI agent or improperly follows the I / O mode requested by the originating AI agent (if such a mode was requested), the generated tool can be analyzed and repaired.

[0193] Then at 1640, the tool is executed by an AI agent or a created AI agent (i.e., the tool-generating AI agent). Alternatively, the originating AI agent invokes the tool-generating AI agent, which then attempts to consume the tool itself (e.g., execute the tool) within its agent loop (and, if necessary, potentially repair the tool using another agent). In some embodiments, the tool includes API definitions and implementations (e.g., executables, other AI agents, RPA, etc.). In some embodiments, the tool is executed directly by the agent loop.

[0194] Constrain valid execution trajectories at point 1645. These can be softly constrained during agent loop execution based on valid execution trajectories provided in the prompts, or hard constrained within the agent loop to verify the validity of tool execution. In some embodiments, the response from tool execution is returned by the AI ​​agent to the caller to provide a final response from tool execution or to update the UI. In some cases, tool execution can be deferred to the caller within the context of the request.

[0195] In some embodiments, the tool is a static server-side tool abstractly defined in the definition of the AI ​​agent, and its calls to the AI ​​agent cannot be altered. In some embodiments, the tool is an abstract definition in the definition of the AI ​​agent that calls a specific server-side tool without describing an API. In some embodiments, an API is provided as part of the call to the AI ​​agent and conforms to the definition of the AI ​​agent. In some embodiments, the tool conforms to a pattern describing what each parameter does, what the allowed values ​​are, what the tool itself does, and when the tool should be used. In some embodiments, the root AI agent (i.e., the originating AI) may request the tool to be generated and potentially executed by another agent only for calling a specific pattern-matching tool (e.g., if the caller does not pass it, or if the root AI agent definition explicitly requests automatic generation). For purely dynamic tools, the AI ​​agent may generate tools that the AI ​​agent wants to have, but these tools are neither mentioned nor passed in the call.

[0196] According to an embodiment of the present invention, Figures 14 to 16 The processing steps performed in the process can be executed by a computer program that encodes instructions for use by (multiple) processors. Figures 14 to 16 The computer program may be embodied on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, hard disk drives, flash memory devices, RAM, magnetic tape, and / or any other such medium or combination of media for storing data. The computer program may include processors (e.g., multiple processors) for controlling a computing system. Figure 11 The computing system 1100 has (multiple) processors 1110) to implement Figures 14 to 16 The coded instructions for all or part of the process steps described herein may also be stored on a computer-readable medium.

[0197] Computer programs can be implemented in hardware, software, or a hybrid manner. A computer program can consist of modules that operatively communicate with each other, and these modules are designed to pass information or instructions to a display. A computer program can be configured to operate on a general-purpose computer, an ASIC, or any other suitable device.

[0198] It is readily understood that the components of various embodiments of the invention, as generally described and illustrated in the accompanying drawings, can be arranged and designed in a variety of different configurations. Therefore, the detailed description of embodiments of the invention as illustrated in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.

[0199] The features, structures, or characteristics of the invention described throughout this specification can be combined in one or more embodiments in any suitable manner. For example, references to "certain embodiments," "some embodiments," or similar language throughout this specification indicate that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Therefore, the appearance of the phrases "in some embodiments," "in some embodiments," "in other embodiments," or similar language throughout this specification does not necessarily refer to the same group of embodiments, and the described features, structures, or characteristics can be combined in one or more embodiments in any suitable manner.

[0200] It should be noted that references to features, advantages, or similar language throughout the specification do not imply that all features and advantages achievable using the invention should be or be present in any single embodiment of the invention. Rather, language referring to features and advantages should be understood to mean that a particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention. Therefore, the discussion of features and advantages throughout the specification, as well as similar language, may, but do not necessarily refer to the same embodiments.

[0201] Furthermore, the features, advantages, and characteristics described in this invention can be combined in one or more embodiments in any suitable manner. Those skilled in the art will recognize that the invention can be practiced without one or more specific features or advantages of a particular embodiment. In other instances, additional features and advantages that may not be present in all embodiments of the invention may be recognized in certain embodiments.

[0202] It will be readily understood by those skilled in the art that the invention discussed above can be implemented with steps in a different order and / or with hardware elements in a configuration different from the disclosed configuration. Therefore, although the invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative constructions will be obvious while remaining within the spirit and scope of the invention. Therefore, reference should be made to the appended claims to determine the limits and scope of the invention.

Claims

1. One or more non-transitory computer-readable media storing one or more computer programs, said one or more computer programs being configured to cause at least one processor to: Initialize the AI ​​agent configured as an automatic generation tool; In response to the successful initialization of the AI ​​agent, the system enters the agent loop for the AI ​​agent; and The tool is executed automatically, wherein The tools include application programming interface (API) definitions and implementation methods.

2. The one or more non-transitory computer-readable media according to claim 1, wherein the implementation of the tool includes an executable file, another AI agent, or Robotic Process Automation (RPA).

3. The one or more non-transitory computer-readable media according to claim 1, wherein the execution of the tool is directly performed by the intelligent agent ring.

4. The one or more non-transitory computer-readable media according to claim 1, wherein the one or more computer programs are further configured to cause the at least one processor to: The AI ​​agent returns a response to the caller, providing the final response from the tool's execution or updating the user interface (UI).

5. The one or more non-transitory computer-readable media according to claim 1, wherein the one or more computer programs are further configured to cause the at least one processor to: Another AI agent is created by the AI ​​agent to generate and / or execute the tool.

6. The one or more non-transitory computer-readable media of claim 1, wherein other AI agents generate the tool, and the one or more computer programs are further configured to cause the at least one processor to: In response to the tool not meeting the requirements of the other AI agent and / or the tool not meeting the input / output pattern requested by the AI ​​agent, the other AI agent automatically analyzes and attempts to repair the generated tool.

7. The one or more non-transitory computer-readable media of claim 1, wherein the other AI agent generates the tool, and the one or more computer programs are further configured to cause the at least one processor to: The AI ​​agent executes the tool generated by the other AI agents.

8. One or more non-transitory computer-readable media according to claim 1, wherein the tool is a static, call-specific, or dynamic caller-side or server-side tool.

9. One or more non-transitory computer-readable media according to claim 1, wherein the tool is a static server-side tool abstractly defined in the definition of the AI ​​agent, and the invocation of the static server-side tool against the AI ​​agent cannot be changed.

10. One or more non-transitory computer-readable media according to claim 1, wherein the tool is an abstractly defined in the definition of the AI ​​agent that invokes a specific server-side tool without describing an application programming interface (API).

11. One or more non-transitory computer-readable media according to claim 10, wherein the API is provided as part of a call to the AI ​​agent and conforms to the definition of the AI ​​agent.

12. One or more non-transitory computer-readable media according to claim 1, wherein the tool conforms to a pattern describing what each parameter does, what the allowed values ​​are, what the tool itself does, and when the tool should be used.

13. The one or more non-transitory computer-readable media of claim 1, wherein the tool is a caller-side tool, and the one or more computer programs are further configured to cause the at least one processor to: The execution of the tool is deferred to the caller within the context of the request.

14. The one or more non-transitory computer-readable media of claim 1, wherein the one or more computer programs are further configured to cause the at least one processor to: The execution of the intelligent agent loop is softly constrained based on the valid execution trajectory provided in the prompt.

15. The one or more non-transitory computer-readable media of claim 1, wherein the one or more computer programs are further configured to cause the at least one processor to: The execution of the intelligent agent loop is hard-constrained by verifying the effectiveness of tool execution within the loop.

16. A computer-implemented method, comprising: The computing system enters the intelligent agent loop targeting artificial intelligence (AI) agents; as well as The tool is executed automatically by the computing system, wherein The tools include application programming interface (API) definitions and implementations, and The tool is static, invokes specific or dynamic caller-side or server-side tools.

17. The computer-implemented method according to claim 16, further comprising: The computing system uses the valid execution trajectory provided in the prompts to softly constrain the execution of the agent loop.

18. The computer-implemented method according to claim 16, further comprising: The computing system hard-constrains the execution of the intelligent agent loop by verifying the effectiveness of tool execution within the loop.

19. One or more computing systems, including: The memory stores computer program instructions; as well as At least one processor, the at least one processor being configured to execute the computer program instructions, wherein the computer program instructions are configured to cause the at least one processor to: Initialize the AI ​​agent configured as an automatic generation tool; In response to the successful initialization of the AI ​​agent, the system enters the agent loop for the AI ​​agent; and The tool is executed automatically, wherein The tool includes application programming interface (API) definitions and implementation methods, which include executable files, another AI agent, or robotic process automation (RPA). The tool is static, invokes specific or dynamic caller-side or server-side tools.

20. One or more computing systems according to claim 19, wherein The tool is a static server-side tool that is abstractly defined in the definition of the AI ​​agent, and the calls made by the static server-side tool to the AI ​​agent cannot be changed. The tool is an abstract definition of calling specific server-side tools in the definition of the AI ​​agent without describing the API, or The API is provided as part of the calls to the AI ​​agent and conforms to the definition of the AI ​​agent.

21. One or more computing systems according to claim 19, wherein the tool conforms to a pattern describing what each parameter does, what the allowed values ​​are, what the tool itself does, and when the tool should be used.

22. One or more computing systems according to claim 19, wherein the computer program instructions are further configured to cause the at least one processor to: The execution of the agent loop is softly constrained based on the valid execution trajectory provided in the prompt; or The execution of the intelligent agent loop is hard-constrained by verifying the effectiveness of tool execution within the loop.

Images