Routing engine for LLM-based digital assistant

Abstract

Techniques for routing utterances of a digital assistant to actions using generative artificial intelligence are disclosed herein. An input query from a user that includes particular data may be received. An action and a set of input reference slots within a pattern associated with the action may be identified based on the input query. The input reference slot may be populated by determining whether one or more parameters may be derived from the particular data and populating the input reference slot with a version of the parameter that conforms to the schema. An execution plan including an action containing the set of populated input reference slots may be sent to an execution engine configured to execute the action to generate a response to the input query.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Application No. 63 / 583,225, filed September 15, 2023, pursuant to 35 USC 119(e), the disclosure of which is incorporated herein by reference in its entirety for all purposes. Technical Field

[0003] This disclosure generally relates to digital assistants, and more particularly, but not exclusively, to techniques for routing user input to actions and associated parameters to generate responses to utterances using digital assistants and large language models. Background Technology

[0004] Artificial intelligence (AI) has a wide variety of applications, with significant evolution in the fields of digital assistants and chatbots. Initially, many users sought immediate responses through instant messaging or chat platforms. Recognizing the potential for engagement, organizations leveraged these platforms to interact with entities such as end users in real-time conversations.

[0005] However, maintaining real-time communication channels with entities through human service personnel has proven costly for organizations. To address this challenge, digital assistants or chatbots (also known as bots) have emerged as a solution that simulates conversations with entities, particularly on the internet. These bots enable entities to interact with users through messaging applications they already use or other applications with messaging capabilities.

[0006] Initially, traditional chatbots relied on predefined skill or intent models, requiring entities to communicate within a fixed set of keywords or commands. Unfortunately, this approach limited the bot's ability to engage intelligently and context-dependently in live conversations, hindering its capacity for natural communication. Entities were constrained by the requirement to use specific commands that the bot could understand, often resulting in difficulties in effectively conveying intent.

[0007] The landscape has changed with the integration of Large Language Models (LLMs) into digital assistants or chatbots. LLMs are deep learning algorithms capable of performing a wide range of Natural Language Processing (NLP) tasks. They utilize a neural network architecture called a transformer, which learns from patterns and structures in natural language and engages in more nuanced and context-aware conversations for various domains and purposes. This evolution marks a significant shift from rigid, keyword-based interactions to a more adaptive and intuitive communication experience compared to traditional chatbots, enhancing the overall ability of digital assistants or chatbots to understand and respond to user queries. Summary of the Invention

[0008] In various embodiments, a computer-implemented method can be used to identify actions and associated parameters to generate an execution plan for responding to a user using a digital assistant. The method may include receiving an input query from a user, wherein the input query includes specific data. The method may include identifying an action among one or more candidate actions based on the input query. The method may include identifying a set of input argument slots within a pattern associated with the action. For each input argument slot in the set, the method may include populating the input argument slot by: determining whether one or more parameters corresponding to the argument slot are derived from the specific data, and, based on the one or more parameters corresponding to the input argument slot, (i) deriving the one or more parameters from the specific data, and (ii) populating the input argument slot with a pattern-compliant version of the one or more parameters. The method may include transmitting an execution plan including the action containing the set of populated input argument slots to an execution engine configured to execute the action to generate a response to the input query.

[0009] In some embodiments, receiving an input query may further include receiving contextual information. Contextual information may include (i) conversation history associated with the user and (ii) historical execution plans. Additionally or alternatively, identifying actions may include identifying actions based on the input query, conversation history, and historical execution plans.

[0010] In some embodiments, identifying actions may also include using a generative artificial intelligence model to select the action to be performed from candidate actions based on the input query, conversation history, and historical execution plans.

[0011] In some embodiments, the method may further include determining that at least one input argument slot in the set of input argument slots cannot be filled using the one or more parameters. The one or more parameters may be missing at least one parameter. The method may further include extracting the at least one parameter from conversation history using a generative artificial intelligence model. The method may further include filling the at least one input argument slot using the at least one parameter.

[0012] In some embodiments, the method may further include determining that at least one input argument slot in the set of input argument slots cannot be filled using the one or more parameters. The version of the one or more parameters may not conform to a pattern. The method may further include adjusting the version of the one or more parameters to conform to the pattern. The method may further include using the adjusted version of the one or more parameters in the pattern to fill the at least one input argument slot.

[0013] In some embodiments, the method may further include determining a first subset of the set of input parameter slots, wherein the first subset of the input parameter slots includes input parameter slots required to perform an action. The method may further include determining a second subset of the set of input parameter slots, wherein the second subset includes input parameter slots optional to perform an action.

[0014] In some embodiments, the method may further include: determining, based on determining that a first subset includes at least one input argument slot that cannot be filled with a version of the one or more parameters, whether context information contained in the input query includes one or more indications of whether the one or more parameters are included. Based on determining that the context information includes one or more indications of whether the one or more parameters are included, the method may further include filling the at least one input argument slot with the version of the one or more parameters. The method may further include: based on determining that the context information does not include a version of the one or more parameters, generating output that can be used to request subsequent input from the user to receive a version of the one or more parameters.

[0015] In some embodiments, the method may further include: determining whether context information contained in the input query includes one or more indications regarding whether the input query includes a version of the one or more parameters, based on determining that the second subset includes at least one input argument slot that cannot be populated with a version of the one or more parameters. The method may further include: populating the at least one input argument slot with the version of the one or more parameters, based on determining that the context information includes a version of the one or more parameters. The method may further include: populating the set of populated input argument slots with empty slots for the at least one input argument slot, based on determining that the context information does not include a version of the one or more parameters, and transferring the set of populated input argument slots to the execution engine.

[0016] In some embodiments, the method may further include using an execution engine and executing an execution plan using the set of populated input argument slots to generate a response to the input query. The method may also include transmitting the response to a user to facilitate interaction involving that user.

[0017] In various embodiments, an apparatus is provided that includes components for implementing some or all of the operations and / or methods disclosed herein.

[0018] In various embodiments, a computer program product is provided that includes computer instructions that, when executed by a processor, implement some or all of the operations and / or methods disclosed herein.

[0019] Some embodiments include a system comprising one or more processors and one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform some or all of the operations and / or methods disclosed herein.

[0020] Some embodiments include a non-transitory computer-readable medium storing one or more instructions that, when executed by one or more processors, cause a system to perform some or all of the operations and / or methods disclosed herein.

[0021] The techniques described above and below can be implemented in a variety of ways and in a variety of contexts. Several example implementations and contexts are provided in the diagram below, as described in more detail below. However, the implementations and contexts below are only a few of the many. Attached Figure Description

[0022] Figure 1 This is a simplified block diagram of a distributed environment incorporating a chatbot system according to various embodiments.

[0023] Figure 2This is an exemplary architecture of a large language model (LLM) based digital assistant according to various embodiments.

[0024] Figure 3 This is a simplified block diagram of a computing environment including a digital assistant according to various embodiments, which can execute an execution plan including actions and associated parameters in response to a user's speech.

[0025] Figure 4 This is a simplified block diagram illustrating a data flow of associated patterns for populating actions using a routing engine, according to various embodiments, for a digital assistant that can execute an execution plan in response to a user's words.

[0026] Figure 5 It is a flowchart of the process of generating an execution plan for a digital assistant by using a routing engine to fill the input argument slots in the associated pattern of the action according to various embodiments.

[0027] Figure 6 This is a block diagram illustrating a pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0028] Figure 7 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0029] Figure 8 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0030] Figure 9 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0031] Figure 10 This is a block diagram illustrating an example computer system according to at least one embodiment. Detailed Implementation

[0032] In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of certain embodiments. However, it will be apparent, however, that various embodiments may be practiced without these specific details. The accompanying drawings and description are not intended to be limiting. The word “exemplary” as used herein means “serving as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as being more preferred or advantageous than other embodiments or designs.

[0033] introduction

[0034] Artificial intelligence (AI) technology has broad applicability. For example, digital assistants can be or include AI-driven interfaces that help users complete various tasks by conversing in natural language. Typically, for each digital assistant, a customer can assemble one or more skills focused on specific types of tasks, such as tracking inventory, submitting attendance cards, and creating expense reports. When an end user interacts with a digital assistant, the assistant evaluates the end user's input against the user's intent and routes conversations to and from appropriate skills based on the perceived user intent. However, traditional intent-based skills have some drawbacks, including limited understanding of natural language, inability to handle unknown input, limited ability to maintain offscript natural conversation, and challenges in integrating external knowledge.

[0035] The emergence of large language models (LLMs) such as GPT-4 has propelled the field of digital assistant design to unprecedented levels of complexity, overcoming these and other shortcomings of traditional intent-based skills. LLMs are neural networks employing a transformer architecture specifically designed to process and generate sequential data, such as text or words from conversations. Trained on large amounts of text data, LLMs gradually hone their ability to generate text that closely mimics human written or spoken language. One of the main advantages of LLMs over traditional language models (LMs) is their ability to generalize more effectively to novel scenarios and domains. Given the inherent flexibility of LLMs, it is hoped that they can be utilized in digital assistants as an agent framework to respond to user queries.

[0036] These agent frameworks are not yet fully mature, but the ultimate agent framework reimagined with LLM will allow for seamless, out-of-the-box human-like conversation, easier asset unlocking, and high-quality routing and orchestration of requests. Current agent frameworks are limited by skill complexity as they need to continue supporting the currently defined ways of acting (intent), knowledge (answer intent), and dialogue (YAML, Freemarker). While routing for digital assistants can be enhanced and some core routing principles remain valid, freely rethinking routing (primarily in the LLM world) and appropriately combining planning, reasoning, and orchestration while considering conventional digital assistant components and architectures has always been a challenge. Nevertheless, to improve and deliver value in the long term, certain digital assistant components and architectures need to be revised or redesigned to make the agent framework LLM-centric; this includes redefining its reasoning / routing engine, composition units, and inherent conversational capabilities.

[0037] To address these and other challenges, routing and planning components and technologies have been incorporated into digital assistants, as detailed in this paper. For each digital assistant, a user can assemble one or more agents. An agent (which may at least partially include one or more large language models (LLMs)) is an independent bot that provides human-like conversational abilities for various types of tasks, such as tracking inventory, submitting attendance cards, updating accounts, and creating expense reports. Agents are primarily defined using natural language. Users (such as developers) can create functional agents by pointing them to assets, such as application programming interfaces (APIs), knowledge-based assets (such as documents, URLs, images, etc.), data repositories, previous conversations, etc. Assets are imported into the agent, and then, because the agent is LLM-based, the user can again use natural language to customize the agent to provide additional API customizations for dialogue and routing / reasoning. Operations performed by the agent are achieved through the execution of one or more actions. Actions can be explicit actions written (e.g., actions created to generate natural language text or audio responses for natural language prompts written for responses such as queries – “What impact does XYZ have on my 401k contribution limit?”), or implicit actions created when importing assets (e.g., actions created for the “Change Contribution” and “Get Contribution” APIs, which are available through API assets and configured to change a user’s 401k contribution).

[0038] Accurately mapping user utterances to actions without proper routing and planning can be challenging. In many instances, executing an action may require a large amount of context-dependent information from the user. For example, APIs often require parameters when invoked and may use specific patterns with set arguments to obtain the desired output. Digital assistants executing such actions may repeatedly prompt the user for missing information based on action failures or may fail to execute complex actions associated with a large amount of contextual information. Routing engines can be used in LLM-based environments to determine the appropriate action to execute in response to a user's utterance from a set of candidate actions. Routing and planning can be implemented by the routing engine to identify, obtain, and provide the input arguments for executing the action. By implementing routing, actions can be executed without repeatedly prompting the user or when the action fails. Understanding the user's preferences and goals when selecting actions without requiring prompts can also help improve the efficiency of digital assistants and reduce the user's burden. This improves the technological field of facilitating digital interaction between users and computing resources such as chatbots. Routing engines can be configured to perform slot filling and collect the information needed to execute actions tailored to the user. In some embodiments, the routing engine can retrieve information from a context that includes searchable conversation history between the user and the digital assistant, historical execution plans, and user profile and preference information.

[0039] In various embodiments, a computer-implemented method can be used to identify actions and associated parameters to generate an execution plan for responding to a user using a digital assistant. The method may include receiving an input query from a user, wherein the input query includes specific data. The method may include identifying an action among one or more candidate actions based on the input query. The method may include identifying a set of input argument slots within a pattern associated with the action. For each input argument slot in the set, the method may include populating the input argument slot by determining whether one or more parameters corresponding to the argument slot are derived from the specific data, and, based on the one or more parameters corresponding to the input argument slot, (i) deriving the one or more parameters from the specific data and (ii) populating the input argument slot with versions of the one or more parameters that conform to the pattern. The method may include transmitting an execution plan containing the action with the set of populated input argument slots to an execution engine configured to execute the action to generate a response to the input query.

[0040] As used herein, when an action is “based on” something, it means that the action is based at least partially on at least a portion of that thing. As used herein, the terms “similarly,” “substantially,” “about,” and “approximately” are defined as substantially, but not necessarily fully specified (and include fully specified), as understood by one of ordinary skill in the art. In any disclosed embodiment, the terms “similarly,” “substantially,” “approximately,” or “approximately” may be replaced with “[percentage]” of the specified content, where the percentage includes 0.1%, 1%, 5%, and 10%.

[0041] Digital assistants and knowledge dialogue

[0042] A bot (also known as an agent, chatbot, chattterbot, or talkbot) that functions as part of or implements a digital assistant is a computer program capable of conversing with an end user. Bots typically respond to natural language messages (e.g., asking questions or commenting) through messaging applications that use natural language messaging. Businesses can use one or more bot systems to communicate with end users via messaging applications. The messaging application (which may be referred to as a channel) can be a preferred messaging application that the end user has already installed and is familiar with. Therefore, the end user does not need to download and install a new application to chat with the bot system. Messaging applications can include, for example, over-the-top (OTT) messaging channels (such as Facebook Messenger, Facebook WhatsApp, WeChat, Line, Kik, Telegram, Talk, Skype, Slack, or SMS), virtual personal assistants (such as Amazon Dot, Echo, or Show, Google Home, Apple HomePod, etc.), mobile, web, and cloud application extensions or plugins that leverage chat capabilities to extend native or hybrid / responsive mobile, web, or cloud applications, or voice-based input (such as devices or applications with interfaces that allow interaction using Siri, Cortana, Google Voice, or other voice input).

[0043] In some examples, a robot system may be associated with a Uniform Resource Identifier (URI). A URI can be used as a string to identify the robot system. A URI can be used as a webhook for one or more messaging applications. A URI may include, for example, a Uniform Resource Locator (URL) or a Uniform Resource Name (URN). The robot system may be designed to receive messages from messaging applications (e.g., Hypertext Transfer Protocol (HTTP) POST request messages). HTTP POST request messages can be directed from the messaging application to the URI. In some embodiments, the message may differ from an HTTP POST request message. For example, the robot system may receive messages from a Short Message Service (SMS). While the discussion herein may refer to communication received by the robot system as a message, it should be understood that a message can be an HTTP POST request message, an SMS message, or any other type of communication between the two systems.

[0044] End users can interact with robotic systems through conversational interactions (sometimes called conversational user interface (UI)), much like human-to-human interactions. In some cases, the interaction may involve the end user saying "hello" to the robot, and the robot responding with "hi" and asking how it can help the end user. In other cases, the interaction may be transactional with, for example, a banking robot, such as transferring funds from one account to another; informational with, for example, an HR robot, such as checking holiday balances; or with, for example, a retail robot, such as discussing returning purchased goods or seeking technical support.

[0045] In some embodiments, a robot system can intelligently handle end-user interactions without interacting with the robot system's administrator or developers. For example, an end-user can send one or more messages to the robot system to achieve a desired goal. Messages may include content such as text, emojis, audio, images, video, or other methods of conveying the message. In some embodiments, the robot system can convert the content into a standardized form (e.g., a Representational State Transition (REST) ​​or API call for an enterprise service with the correct parameters) and generate a natural language response. The robot system may also prompt the end-user for additional parameters or request other additional information. In some embodiments, the robot system may also initiate communication with the end-user rather than passively responding to end-user utterances. This document describes various techniques for identifying explicit calls to a robot system and determining the input of the called robot system. In some embodiments, explicit call analysis is performed by the master robot based on the call name detected in the utterance. In response to the detection of the call name, the utterance can be refined or preprocessed for input to the robot identified as associated with that call name and / or communication.

[0046] Figure 1This is a simplified block diagram of an environment 100 incorporating a digital assistant system (also referred to herein as a digital assistant, or more specifically as an agent assistant, with reference to implementations of an agent) according to certain embodiments. Environment 100 includes a Digital Assistant Builder Platform (DABP) 105, which enables a user 110 to create and deploy the digital assistant system 115. For the purposes of this disclosure, a digital assistant is an entity that assists a user of the digital assistant in performing various tasks through natural language conversation. The DABP and digital assistant can be implemented using software alone (e.g., a digital assistant is a digital entity implemented using programs, code, or instructions executable by one or more processors), hardware, or a combination of hardware and software. In some instances, environment 100 is part of an Infrastructure as a Service (IaaS) cloud service (described in detail below) and can be implemented as part of the IaaS by leveraging scalable computing resources and storage capabilities provided by the IaaS provider to handle and manage large amounts of data and complex computations. This setup allows the DABP and digital assistant to provide real-time responsive interaction while ensuring high availability, security, and performance scalability to meet evolving demand levels. Digital assistants can be implemented or realized in various physical systems or devices, such as computers, mobile phones, watches, appliances, vehicles, etc. Digital assistants are sometimes also referred to as chatbot systems. Accordingly, for the purposes of this disclosure, the terms digital assistant and chatbot system are used interchangeably.

[0047] The DABP 105 can be used to create one or more digital assistant systems (or DAs). For example, such as... Figure 1 As shown, user 110 of a specific business can use DABP 105 to create and deploy digital assistants 115A for that specific business's users. For example, a bank can use DABP 105 to create one or more digital assistants for its customers to use, for example, to change 401k payments. Multiple businesses can use the same DABP 105 platform to create digital assistants. As another example, a restaurant owner, such as a pizzeria, can use DABP 105 to create and deploy digital assistants 115B that enable the restaurant's customers to order food (e.g., order pizza).

[0048] To create one or more digital assistant systems 115, the DABP 105 is equipped with a set of tools 120 that enable the acquisition of LLMs, the creation of agents, the identification of assets, and the communication via computing platforms (such as those related to...). Figure 6-10(A detailed description of the cloud computing platform) deploys digital assistant systems within a user-facing service architecture. In some instances, tool 120 can be used to access pre-trained and / or fine-tuned LLMs from a data repository or computing system. Pre-trained LLMs serve as foundational elements, possessing extensive language understanding derived from massive datasets. This capability enables models to generate coherent responses across a wide range of topics, facilitating transfer learning. Pre-trained models offer cost-effectiveness and flexibility, allowing for scalable improvements and continuous pre-training with new data, typically establishing benchmarks in natural language processing (NLP) tasks. Conversely, fine-tuned models are trained specifically for a particular task or industry (e.g., leveraging the contextual learning capabilities of LLMs for plan creation, knowledge or information retrieval on behalf of an agent, generating responses for human-like conversations, etc.), enhancing their performance on specific applications and enabling efficient learning from smaller, specialized datasets. Fine-tuning offers advantages such as task specialization, data efficiency, faster training times, model customization, and resource efficiency. In some embodiments, fine-tuning may be particularly advantageous for niche applications and continuous enhancement.

[0049] In other instances, tool 120 can be used for pre-training and / or fine-tuning LLMs. Tool 120, or any subset thereof, can be a standalone machine learning operational framework, including hardware components such as processors (e.g., CPUs, GPUs, TPUs, FPGAs, or any combination thereof), memory, and storage devices. This framework operates software or computer program instructions (e.g., TensorFlow, PyTorch, Keras, etc.) to execute arithmetic, logical, and input / output commands for training, validating, and deploying machine learning models in production environments. In some instances, tool 120 uses cloud platforms such as Oracle Cloud Infrastructure (OCI) for model training, validation, and deployment. Leveraging cloud platforms makes machine learning more accessible, flexible, and cost-effective, enabling developers to develop and deploy models more quickly.

[0050] Tool 120 also includes a prompt-based agent composition unit for creating agents and their associated actions that end users can ultimately invoke. An agent is a container for agent actions and can be part of one or more digital assistants. Each digital assistant can contain one or more agents through digital assistant relationships, which are cross-entities linking agents to digital assistants. Agents and digital assistants are implemented as robot subtypes and can be persisted to an existing BOTS table. This offers advantages in reusing design-time code (e.g., Java code) and UI artifacts.

[0051] Agent actions belong to a specific action type (e.g., knowledge, service, or API, LLM, etc.) and contain a schema (e.g., a JSON schema) that describes and defines action parameters. The action description and parameter schema are indexed by a semantic index and sent to the planner to select the appropriate action(s) to execute. Action parameters are key-value pairs that serve as inputs to the action execution. They originate from properties in the schema but may also include additional UI / dialogue properties for slot-filling dialogues. Actions can be part of one or more classes. For example, some actions can be part of an application event subscription class that defines the agent action to be performed when an application event is received. Application events can be received as command messages that update the application context. An application event property mapping class (part of the application event subscription class) specifically maps application event payload properties to corresponding agent action parameters. Actions can optionally be part of an action group. Action groups can be used when importing plugin manifests or external API specifications (such as the Open API specification). Action groups are particularly useful when re-importing plugins or open API specifications, allowing you to add new actions, update existing actions, or remove actions from new manifests or open API specifications that no longer exist. At runtime, action groups may only be used to restrict the application context groups sent to the LLM as conversation contexts by looking up the action group name corresponding to the context group context.

[0052] An agent (e.g., a 401k "change payment" agent) can be primarily defined as an assembly of agent artifacts using natural language within a prompt-based agent composition unit. User 110 can quickly create functional agents by providing agent artifact information, parameters, and configuration, and by pointing to assets. Assets can be or include resources such as APIs for interfacing with applications, files and / or documents for retrieving knowledge, data repositories for interacting with data, etc., which can be used by the agent to perform actions. Once assets are imported, User 110 can then again provide additional API customizations using natural language for dialogue and routing / reasoning. Much of what the agent performs likely involves performing actions. Actions can be explicit actions written in natural language (similar to creating agent artifacts—for example, the "What impact does XYZ have on my 401k contribution limit?" action in the "401k Contribution Agent" diagram below), or implicit actions created when importing assets (automatically imported based on metadata and / or specifications associated with the asset when pointing to a given asset—for example, the actions created for the "Change Contribution" and "Get Contribution" APIs in the "401k Contribution Agent" diagram below). Users can easily create explicit actions at design time. For example, when a user learns that a new FAQ needs to be added (because it is not currently in the knowledge document (asset) referenced by the agent (and therefore not implicitly added as an action)), the user can select a "Rich Text" action type (see the list of exemplary action types in Table 1) and create the artifact named "What impact does XYZ have on my 401k contribution limit?".

[0053] Table 1:

[0054]

[0055] There are various ways to associate or add agents and assets to a digital assistant 115. In some instances, agents can be developed by an enterprise and then added to the digital assistant using DABP 105. In other instances, agents can be developed and created using DABP 105 and then added to a digital assistant created using DABP 105. In still other instances, DABP 105 provides an online digital store (called the "Agent Store") that offers a variety of pre-created agents for a wide range of tasks and actions. Agents offered through the Agent Store can also expose various cloud services. To add an agent to a digital assistant created using DABP 105, a user 110 of DABP 105 can access assets via tool 120, select specific assets for the agent, initiate several simulated chat conversations with the agent, and instruct the agent to be added to the digital assistant created using DABP 105.

[0056] Once deployed in a production environment (such as regarding...) Figure 2In the described architecture, a digital assistant (such as digital assistant 115A) built using DABP 105 can be used to perform various tasks via natural language-based conversation between digital assistant 115A and its user 125. As described above, this enables... Figure 1 The digital assistant 115A shown is available for use or access by its user 125 through a variety of different channels, such as, but not limited to, certain applications, social media platforms, various messaging services and applications, and other applications or channels. A single digital assistant may have several channels configured for it, allowing it to run and be accessed by different services simultaneously.

[0057] As part of a conversation, user 125 can provide one or more user inputs 130 to digital assistant 115A via a user interface element such as a chat window and receive responses 135 from digital assistant 115A. A conversation can include one or more of user inputs 130 and responses 135. Through these conversations, user 125 can request one or more tasks to be performed by digital assistant 115A, and in response, digital assistant 115A is configured to perform the user-requested task and respond to user 125 with an appropriate response using one or more LLMs 140. Conversations displayed in the chat window can be organized by thread. For example, in some applications, conversations related to one page of the application should not be mixed with conversations related to another page of the application. Applications and / or plugins of the application define thread boundaries (e.g., a set of (nested) plugins can run within their own threads). In practice, the chat window will only display the history of messages belonging to the same thread. Thread settings and changes can be performed via the application and / or plugins using update application context command messages. Additionally or alternatively, when a user query matches a plugin semantic action and the plugin is running in a thread different from the current thread, the thread can be changed via the execution plan orchestrator. In this case, the planner changes the thread so that any messages sent in response to the executed action are displayed in the correct new thread. For each agent dialogue thread, the digital assistant can maintain the following information: application context, LLM conversation history, conversation history with the user, and agent execution context that keeps information about one or more (stacked) execution plans associated with that thread.

[0058] User input 130 is typically in natural language form and is referred to as a utterance (which may also be referred to as a prompt, query, request, etc.). User input 130 can be in text form, such as when a user types a sentence, question, text fragment, or even a single word and provides it as input to the digital assistant 115A. In some embodiments, user input 130 can be in audio input or speech form, such as when a user speaks or utters something provided as input to the digital assistant 115A. User input 130 is typically in the language spoken by the user 125. For example, user input 130 might be English or some other language. When user input 130 is in speech form, the speech input is converted into text form of user input 130 in that particular language, and then the text utterance is processed by the digital assistant 115A. Various speech-to-text processing techniques can be used to convert speech or audio input into text utterance, which is then processed by the digital assistant 115A. In some embodiments, the speech-to-text conversion can be performed by the digital assistant 115A itself. For the purposes of this disclosure, it is assumed that user input 130 is either text speech directly provided by user 125 of digital assistant 115A, or the result of inputting voice speech converted into text form. However, this is not intended to limit or constrain in any way.

[0059] User input 130 can be used by digital assistant 115A to determine a list of candidate agents 145A-N. The list of candidate agents (e.g., 145A-N) includes agents configured to perform one or more actions that may potentially facilitate a response 135 to user input 130. This list can be determined by running a search (such as a semantic search) on a context and memory repository having one or more indices containing metadata of all agents 145 available to digital assistant 115A. The metadata of candidate agents 145A-N in the list of candidate agents is then combined with the user input to construct an input prompt for one or more LLMs 140.

[0060] The digital assistant 115A is configured to apply NLP techniques to text and / or speech using one or more LLMs 140 to understand input prompts and to apply Natural Language Understanding (NLU), including syntactic and semantic analysis of the text and / or speech, to determine the meaning of the user input 130. Determining the meaning of the utterance may involve identifying the user's goal, one or more of the user's intentions, the context surrounding various words or phrases or sentences, one or more entities corresponding to the utterance, etc. NLU processing may include parsing the received user input 130 to understand the structure and meaning of the utterance, refining and reshaping the utterance to develop a more understandable form (e.g., logical form) or structure. The NLU processing performed may include various NLP-related processes such as sentence parsing (e.g., lexicalization, lemmatizing, identifying part-of-speech tags of sentences, identifying named entities in sentences, generating dependency trees to represent sentence structure, splitting sentences into clauses, analyzing individual clauses, parsing pronouns, performing chunking, etc.). In some instances, NLU processing, or any part thereof, is performed by the LLM 140 itself. In other instances, the LLM 140 uses additional resources to perform parts of the NLU processing. For example, the syntax and structure of the input utterance sentence can be identified by processing the sentence using parsers, part-of-speech taggers, named entity recognition models, pre-trained language models (such as BERT), etc.

[0061] After understanding the meaning of the utterance, one or more LLMs 140 generate an execution plan that identifies one or more agents (e.g., agent 145A) from a list of candidate agents to perform and conduct one or more actions or operations in response to the understood user meaning or goal. The one or more actions or operations are then executed by the digital assistant 115A on one or more assets (e.g., asset 150A—knowledge, API, SQL operations, etc.) and / or a context and memory repository. The execution of the one or more actions or operations generates output data from the one or more assets and / or generates relevant context and memory information from a context and memory repository that includes the context of the current conversation with the digital assistant 115A. The output data and relevant context and memory information are then combined with user input 130 to construct an output prompt for one or more LLMs 140. The LLM 140 synthesizes a response 135 to user input 130 based on the output data, relevant context and memory information, and user input 130. The response 135 is then sent to user 125 either as a separate response or as part of a conversation with user 125.

[0062] For example, user input 130 could request to order pizza by providing a utterance such as “I want to order pizza.” Upon receiving such a utterance, digital assistant 115A is configured to understand the meaning or goal of the utterance and take an appropriate action. The appropriate action might involve, for example, providing the user with a response 135 and requesting the user to input questions about the type of pizza the user wishes to order, the size of the pizza, any toppings, etc. The user's request for questions can be generated by performing actions on knowledge assets (e.g., a pizza restaurant's menu) via an agent (e.g., agent 145A) to retrieve information related to ordering pizza (e.g., to order pizza, the user must provide the type, size, toppings, etc.). The response 135 provided by digital assistant 115A can also be in natural language form and is typically the same language as user input 130. As part of generating these responses 135, digital assistant 115A may perform natural language generation (NLG) using one or more LLMs 140. For a user ordering pizza, the digital assistant 115A can guide the user to provide all the necessary information for the pizza order through a conversation between the user and the digital assistant 115A, and then the pizza is ordered at the end of the conversation. The order can be executed by an agent (e.g., agent 145A) performing an action on an API asset (e.g., an API for ordering pizza) to upload or provide the pizza order to the restaurant's ordering system. The digital assistant 115A can end the conversation by generating a final response 135 to the user 125 indicating that the pizza has been ordered.

[0063] While the various examples provided in this disclosure describe and / or illustrate utterances in the English language, these are merely examples. In some embodiments, the digital assistant 115 may also handle utterances in languages ​​other than English. The digital assistant 115 may provide subsystems (e.g., components implementing NLU functionality) configured to perform processing for different languages. These subsystems may be implemented as pluggable units that can be invoked using service calls from the NLU core server. This makes NLU processing flexible and scalable for each language, including allowing different processing sequences. Language packs may be provided for each language, wherein the language packs may register a list of subsystems that can be serviced by the NLU core server.

[0064] Although Figure 1 The embodiments illustrated herein include a digital assistant 115A comprising one or more LLMs 140 and one or more agents 145A-N, but this is not intended to be limiting. The digital assistant may include various other components that provide the functionality of a digital assistant (e.g., as per [reference to...]). Figure 2Other systems and subsystems (more detailed description of other systems and subsystems). The digital assistant 115A and its systems and subsystems may be implemented using software only (e.g., code or instructions stored on a computer-readable medium and executable by one or more processors), hardware only, or a combination of software and hardware.

[0065] Figure 2 This is an example of the architecture of a computing environment 200 using generative artificial intelligence, according to various embodiments. For example... Figure 2 As shown, the infrastructure, along with various services and features, can be used to enable users to interact with digital assistants (e.g., regarding...) at least in part based on a series of prompts (such as conversation) and / or a series of actions (such as interaction with the user interface). Figure 1 The interaction takes place with the digital assistant 115A described below. The following is a detailed walkthrough of the conversation flow, and the roles and responsibilities of the components, services, models, etc., of the computing environment 200 within the conversation flow. In this walkthrough, assume that user "David" is interested in making changes to his 401k contribution, and in utterance 202, David provides the following input: Hi, how are you? I want to change my 401k contribution. utterance 202 can be transmitted to the digital assistant (e.g., via a digital assistant user interface such as a text dialog box or microphone). Upon receiving utterance 202 at this stage, the sessionist creates a new session or retrieves the current session context, and the user message publisher updates the session transcript and LLM message history with the new user message (e.g., utterance 202).

[0066] In instances where a user provides utterances (202) and / or performs actions while using an application supported by a digital assistant, the application issues an update application context command when the user interacts with the application (e.g., providing utterances via text or audio, triggering user interface elements, navigating between pages in the application, etc.). Whenever the digital assistant receives an update application context command message from the application, an application context handler (part of the context manager) is implemented. The application context handler performs the following tasks: (i) managing conversation threads based on the application context message, for example, creating a new conversation thread and making it the current thread if the threadId specified in the message does not already exist, and making the corresponding conversation thread the current thread if the threadId already exists; (ii) creating or updating application context objects for the current conversation thread; and (iii) enriching the application context (as described in more detail herein) if a service call ID, such as a REST request ID, is included. It should be understood that the application context only contains information reflecting the state of the application's user interface and plugins (if available); it does not contain other state information (e.g., user or page state information / context).

[0067] In some instances, when an update application context command message is received, the application event handler checks whether the update application context command message includes an event definition. An event is uniquely identified by the following attributes in the message payload: (i) context: the context path and / or plugin path (for top-level workspace plugins, the context is set to the plugin name; for nested plugins, it includes the plugin paths separated by slashes, e.g., Patient / Vitalschart); (ii) eventType: the type of the event, which can be either a built-in event or a custom event; and (iii) semantic object: the semantic object to which the event is applied. Events can be mapped to one or more actions, and message payload attributes can be mapped to action parameters. This mapping is done through the application event subscription. Each attribute in the message payload can be mapped to an agent action parameter using the application event attribute map.

[0068] In some instances, utterance 202 and / or user-performed actions are provided directly as input to routing engine 208 (also known as the planner). In other instances implementing an application event processor, utterance 202 and / or user-performed actions are provided as input to routing engine 208 when the application event processor determines that an event such as receiving utterance 202 maps to an agent or action associated with the digital assistant. Routing engine 208 is used by the digital assistant to create execution plan 210 using specified parameters from utterance 202, user-performed actions, context, or any combination thereof. Execution plan 210 identifies one or more agents and / or one or more actions to be performed by one or more agents in response to utterance 202 and / or user-performed actions.

[0069] An execution plan 210 can be generated in two steps via routing engine 208. First, a search 212 can be performed to identify a list of candidate agents and / or actions. Search 212 involves running queries against an index 213 (e.g., a semantic index) of context and memory repository 214 based on utterances 202 and / or user-performed actions. In some instances, search 212 is a semantic search performed using words in utterances 202 and / or that represent user-performed actions. Semantic search uses NLP and optional machine learning techniques to understand the meaning of utterances 202 and / or user-performed actions and retrieves relevant information from context and memory repository 214. Compared to traditional keyword-based searches that rely on exact matches between words in the query and data in context and memory repository 214, semantic search considers relationships between words, the context of the query and / or actions, synonyms, and other linguistic nuances. This allows the digital assistant to provide more accurate and context-sensitive results, making it more effective in understanding user intent in utterances 202 and / or user-performed actions.

[0070] To run the query, routing engine 208 invokes context and memory repository 214 (e.g., the semantic index of context and memory repository 214) to obtain a list of candidate agents and / or actions. The following information is passed in the call: (i) the digital assistant's ID (which defines the set of agents and / or actions the semantic index will search, and therefore the agents and / or actions must be part of the digital assistant), and (ii) the last X user messages and / or actions (e.g., X can be set to the last 5 rounds), which can be configured via the digital assistant settings. Upon receiving the list of candidate agents and / or actions, routing engine 208 can identify patterns associated with those actions and perform slot filling to determine any missing input arguments for that pattern.

[0071] The context and memory repository 214 is implemented using a data framework that connects external data to the LLM 216, allowing users to easily plug in custom data sources. The data framework provides a rich and efficient retrieval mechanism on top of data from various sources, such as files, documents, data repositories, APIs, etc. Data can be external (e.g., enterprise assets) and / or internal (e.g., user preferences, memories, digital assistant and agent metadata, etc.). In some instances, the data includes metadata extracted from artifacts 217 associated with the digital assistant and its agents 218 (e.g., 218a and 218b). The digital assistant's artifacts 217 include information about the general capabilities of the digital assistant, as well as specific information about the capabilities (e.g., actions) of each agent 218 (e.g., agent artifacts) available to the digital assistant. Additionally or alternatively, artifacts 217 may contain parameters or information associated with artifacts 217 and that can be used to define agents 218, wherein the parameters or information associated with artifacts 217 may include a name, description, one or more actions, one or more assets, one or more customizations, etc. In some instances, the data also includes metadata extracted from assets 219 associated with the digital assistant and its agents 218 (e.g., 218a and 218b). Assets 219 can be resources that agents 218 can use to perform actions (e.g., actions 225a, 225b, and 225c), such as APIs 220, files and / or documents 222, data repositories 223, etc. The data is indexed in a context and memory repository 214 as an index 213, which is a data structure that provides a fast and efficient way to find and retrieve specific data records within the data. Therefore, the context and memory repository 214 provides a searchable and comprehensive record of all the capabilities available to the digital assistant to respond to requests and / or actions by agents and associated assets.

[0072] The responses from the context and memory repository 214 are transformed into a list of agent and / or action instances that are not only available to the digital assistant in response to requests but also potentially facilitate the generation of responses to utterance 202 and / or user-performed actions. The list of candidate agents and / or actions contains metadata associated with each candidate agent and / or action in the context and memory repository 214 (e.g., metadata extracted from artifact 217 and asset 219). This list may be limited to a predetermined number of candidate agents and / or actions that satisfy the query (e.g., the top 10), or it may include all agents and / or actions that satisfy the query. The list of candidate agents and / or actions with associated metadata is appended to utterance 202 and / or user-performed actions to construct input prompts 227 for LLM 216. Search 212 is important for the digital assistant because it filters out agents and / or actions that are unlikely to facilitate the generation of responses to utterance 202 and / or user-performed actions. This filtering ensures that the number of tokens (e.g., word tokens) generated from input prompt 227 remains within the maximum token limit or contextual limit set for LLM 216. The token limit represents the maximum amount of text that can be input into the LLM. This limit is technical in nature and arises from computational constraints such as memory and processing resources, and thus ensures that the LLM can accept input prompts as input.

[0073] In some instances, one or more knowledge actions are additionally appended to the list of candidate agents and utterance 202. Knowledge actions allow for the acquisition of additional knowledge related to utterance 202 and / or actions performed by the user (this knowledge is often outside the knowledge scope of the LLM used to train the digital assistant). There are two types of knowledge action sources: (i) structured: the knowledge source defines a list of predefined questions that the user may ask and exposes it as some API (e.g., Multum), and (ii) unstructured: with the knowledge source, the user has an unlimited number of ways to ask questions, and the knowledge source exposes a general query interface (e.g., medical documents (SOAP notes, discharge summaries, etc.)).

[0074] In some instances, conversation context 229 regarding utterance 202 is additionally appended to the list of candidate agents and utterance 202. Conversation context 229 can be retrieved from one or more sources, including context and memory repositories 214, and includes user session information, conversation state, conversation or context history, application context, page context, user information, or any combination thereof. For example, conversation context 229 may include: the current date and time (required to resolve time references in user queries, such as "yesterday" or "next Thursday"), additional context (which contains information such as user profile attributes and application context groups with semantic object attributes), and chat history with the digital assistant (and / or other digital assistants or systems internal or external to computing environment 200).

[0075] The second step of the two-step process involves LLM 216 generating an execution plan 210 based on input prompt 227. The invocation of LLM 216 can be performed by: creating an LLM chat message with a role system; receiving input prompt 227; converting candidate agents and / or actions into LLM function definitions; retrieving the correct LLM client based on LLM configuration options; optionally transforming input prompt 227, the LLM chat message, etc., into the correct format for the LLM client; and sending the LLM chat message to the LLM client to invoke LLM 216. The LLM client then sends back an LLM success response in CLMI format, or uses an adapter such as OpenAIAdapter to convert a provider-specific response back to a CLMI format LLM success response (or sends back or converts to an LLM error response in the event of an unexpected error). An LLM invocation instance is created and added to the conversation history, which captures all request and response details, including execution time.

[0076] LLM 216 employs a deep generative model architecture (e.g., an invertible or autoregressive architecture) to generate execution plans 210. In some instances, LLM 216 has over 100 billion parameters and uses autoregressive language modeling within a transformer architecture to generate execution plans 210, allowing LLM 216 to capture complex patterns and dependencies in input cues 227. LLM 216's ability to generate execution plans 210 is a result of its training on diverse and extensive textual data, enabling LLM to understand human language in various contexts. During training, LLM 216 learns to predict the next word in a sequence based on the context of the preceding word. This process involves adjusting the model's parameters (weights and biases) based on the error between its prediction and the actual next word in the training data. When LLM 216 receives input such as input cues 227, it breaks down the text into smaller units, such as words or sub-words. Each word is then represented as a vector in a multi-dimensional space. The LLM 216 processes the input sequence word by word, maintaining an internal representation of the context. Its attention mechanism allows it to measure the importance of different words in the context of generating the next word. For each word in the vocabulary, the LLM 216 computes a probability distribution based on its learned parameters. This probability distribution represents the likelihood that each word will be the next word in a given context. For example, to generate execution plan 210, the LLM 216 samples words from the computed probability distribution. The sampled word becomes the next word in the generated sequence. This process is repeated iteratively, with each newly generated word influencing the context for generating subsequent words. The LLM 216 can continue generating words until a predefined length or stopping condition is reached.

[0077] In some instances, such as Figure 2As shown, LLM 216 may fail to generate a complete execution plan 210 because it lacks information such as whether more information is needed to determine the appropriate agent for the response, to perform one or more actions, etc. In this particular instance, LLM 216 has determined that, in order to change a 401k contribution at the user's request, it is necessary to know whether the user is willing to change the contribution by a percentage or a specific monetary amount. To obtain this information, LLM 216 (or another LLM such as LLM 236) generates an end-user response 235 to input prompt 227 (I'm fine. Would you like to change your contribution by a percentage or an amount? [Percentage] [Amount]), which obtains the missing information, allowing LLM 216 to generate a complete execution plan 210. In some instances, the response can be rendered within a dialog box in the UI, which has one or more UI elements to allow the user to respond more easily. In other instances, the response can be rendered within a dialog box in the UI, allowing the user to respond using the dialog box (or an alternative, such as a microphone). In this particular instance, the user responds with an additional query 238 (What is my current 401k contribution? And can you tell me my contribution limit?) to gather additional information, allowing the user to reply with response 235. The subsequent response—additional query 238—is fed into the routing engine 208 and undergoes the same processing described above for utterance 202, but this time with context from the user's conversation with the digital assistant regarding previous utterances / responses (e.g., utterance 202 and response 235). This time, as... Figure 2 As shown, LLM216 is able to generate a complete execution plan 210 because it has all the information it needs.

[0078] In some instances, a user's utterance 202 can be identified by LLM 216 as an illogical utterance (i.e., an utterance that logically does not follow the previous utterance in a conversation or dialogue). In such instances, an execution plan orchestrator can be used to handle the switching between different conversation paths. The execution plan orchestrator is configured to: track all ongoing conversation paths; if a new conversation path is created, create a new entry and pause the currently ongoing conversation (if any); if the conversation is completed, remove the entry based on metadata or user preferences of the new action; it may generate a prompt message when an illogical utterance begins or continues a previous utterance; manage the conversation for which a prompt message is sent and choose to continue or resume the current conversation; confirm or cancel when the user responds to a prompt for an illogical utterance; and manage cancellation or exit from the conversation.

[0079] Execution plan 210 includes an ordered list of agents and / or actions that can be used and / or executed to adequately respond to requests (such as additional query 238). For example, and as... Figure 2As shown, execution plan 210 may be an ordered list including a first agent 242a capable of performing a first action 244a via an associated asset and a second agent 242b capable of performing a second action 244b via the associated asset. The agents and the actions extended therefrom may be ordered such that the first agent 242a performs the first action 244a before the second agent 242b performs the second action 244b. In some instances, execution plan 210 may be ordered based on dependencies indicated by the agents and / or actions included in it. For example, if the execution of the second agent 242b depends on or otherwise requires the output generated by the first agent 242a performing the first action 244a, then execution plan 210 may command the first agent 242a and the second agent 242b to comply with that dependency. It should be understood that other examples of dependencies are also possible.

[0080] The execution plan 210 is then transmitted to the execution engine 250 for implementation. The execution engine 250 includes several engines, including a natural language to programming language translator 252, a knowledge engine 254, an API engine 256, a prompting engine 258, etc., for executing the agent's actions and implementing the execution plan 210. For example, the natural language to programming language translator 252 (such as a conversation to Oracle Meaning Representation Language (C2OMRL) model) can be used by the agent to translate natural language into (e.g., OMRL) intermediate logic, convert that intermediate logic into a system programming language (e.g., SQL), and execute that system programming language (e.g., execute an SQL query) on asset 219 (such as a data repository 223) to perform actions and / or obtain data or information. The knowledge engine 254 can be used by the agent to obtain data or information from context and memory repositories 214 or asset 219 (such as files / documents 222). The API engine 256 can be used by the agent to call API 220 and interface with applications (such as retirement account management applications) to perform actions and / or obtain data or information. The prompting engine 258 can be used by an agent to construct prompts for input into an LLM (such as an LLM in a context and memory repository 214 or an asset 219) to perform actions and / or obtain data or information.

[0081] Execution engine 250 implements execution plan 210 by using one or more appropriate engines to run each agent and execute each action sequentially based on an ordered list of agents and / or actions. To facilitate this implementation, execution engine 250 communicates with agents (e.g., 242a, 242b, etc.), context and memory repository 214, and assets 219 (e.g., via public and / or private networks). Figure 2As shown, when execution engine 250 implements execution plan 210, it will first use API engine 256 to execute agent 242a and action 244a to call API 220 and interface with the retirement account management application to retrieve the user's current 401k contributions. Subsequently, execution engine 250 can use knowledge engine 254 to execute agent 242b and action 244b to retrieve knowledge about 401k contribution limits. In some instances, knowledge engine 254 retrieves knowledge from asset 219 (e.g., file / document 222). In other instances (as in this particular instance), knowledge is retrieved by knowledge engine 254 from context and memory repository 214. Various techniques can be used to implement knowledge retrieval and action execution using context and memory repository 214, including internal task mapping and / or machine learning models, such as attached LLM models. For example, a query for “What is the 401k contribution limit?” and the associated agent can be mapped to a “semantic search” knowledge task type, used to search for a response to a given query in index 213 within the context and memory repository 214. As another example, a request such as “Can you summarize the key points related to 401k contributions?” can be or include a “summarization” knowledge task type, which can be mapped to different indexes within the context and memory repository 214, which have LLMs trained to create natural language responses (e.g., summaries of key points related to 401k contributions) for a given query. Over time, a library of general end-user task or action types (e.g., semantic search, summarization, comparison / contrast, heterogeneous data synthesis, etc.) can be built to ensure that the indexes and models within the context and memory repository 214 are optimized for various task or action types.

[0082] The result of executing execution plan 210 is output data 269 (e.g., the outcome of an action, data, information, etc.), which is transmitted to output pipeline 270 to generate end-user response 272. For example, output data 269 (knowledge, API, conversation history, etc.) from assets 219, along with relevant information from context and memory repository 214, can be transmitted to output pipeline 270. Output data 269 is appended to utterance 202 to construct output prompt 274 for input to LLM 236. In some instances, context 229 about utterance 202 is appended to both output data 269 and utterance 202. Context 229 can be retrieved from context and memory repository 214 and includes user session information, conversation state, conversation or context history, user information, or any combination thereof. LLM 236 generates response 272 based on output prompt 274. In some instances, LLM 236 is the same or similar model to LLM 216. In other instances, LLM 236 differs from LLM 216 (e.g., trained on different datasets, with different architectures, trained for one or more different tasks, etc.). In any instance, LLM 236 has a deep generative model architecture for generating responses 272 using similar training and generative processing as described with respect to LLM 216 (e.g., with an invertible or autoregressive architecture). In some instances, LLM 236 has over 100 billion parameters and uses autoregressive language modeling to generate responses 272 within the transformer architecture, allowing LLM 236 to capture complex patterns and dependencies in the output cues 274.

[0083] In some instances, the end-user response 272 can be formatted using the Conversational Message Model (CMM) and output as a rich, multimodal response. The CMM defines various message types that a digital assistant can send to a user (outbound) and that a user can send to a digital assistant (inbound). In some instances, the CMM recognizes the following message types:

[0084] Text: Basic text message

[0085] Cards: Card representations that include a title and optional descriptions, images, and links.

[0086] Attachment: Message with media URL (file, image, video, or audio)

[0087] Location: A message containing geographic location coordinates.

[0088] Postback: A message with a postback payload

[0089] Messages defined in CMM are channel-independent and can be created using CMM syntax. Channel-specific connectors transform CMM messages into the format required for a specific channel, allowing users to run digital assistants on multiple channels without having to create separate message formats for each channel.

[0090] Finally, output pipeline 270 transmits response 272 to the end user (e.g., via user device or interface). In some instances, response 272 is rendered within a dialog box in the GUI, allowing the user to view and respond using the dialog box (or alternative means, such as a microphone). In other instances, response 272 is rendered within a GUI dialog box with one or more GUI elements, allowing for easier user responses. In this particular instance, the first response 272 to the additional query 238 (What is my current 401k contribution? Also, could you tell me my contribution limit?) is rendered within a dialog box in the GUI. Furthermore, to follow up on information still needed to obtain the initial statement 202, LLM 236 generates another response 272 prompting the user to provide the missing information (Would you like to change your contribution by percentage or amount? [Percentage] [Amount]).

[0091] Although Figure 2 The embodiments of computing environment 200 illustrate the interaction of a digital assistant in a specific conversation flow, but this is not intended to be limiting, but merely to facilitate a better understanding of the role and responsibilities of the components, services, models, etc. of computing environment 200 within the conversation flow.

[0092] Block diagram of the computing environment including the routing engine for digital assistants

[0093] Figure 3 It is a computing environment including a digital assistant 300 according to various embodiments (e.g., reference 1). Figure 2A simplified block diagram of the described computing environment 200 is shown. The digital assistant 300 can execute an execution plan including actions and associated parameters in response to a user's speech. In some embodiments, speech may be provided to the digital assistant 300 from the user via input 302. Input 302 may be or include natural language speech, which may include text input, voice input, image input, or any other suitable input for the digital assistant 300. For example, input 302 may include text input provided by the user via the keyboard or touchscreen of the computing device they are using. In other examples, input 302 may include spoken words provided by the user via the microphone of the computing device. In other examples, input 302 may include image data, video data, or other media provided by the user via the computing device. Additionally or alternatively, input 302 may include instructions for actions to be performed by the digital assistant 300 on behalf of the user. For example, input 302 may include instructions for the user to order pizza, instructions for the user to renew retirement account contributions, or other suitable instructions.

[0094] Input 302 can be provided to routing engine 304. Routing engine 304 can generate an execution plan based on input 302 and the context provided to routing engine 304. Routing engine 304 can receive input 302 and can invoke semantic context and memory repository 306 to retrieve the context. In some embodiments, semantic context and memory repository 306 includes one or more assets 308, which may be similar to or the same as asset 219. Routing engine 304 can provide at least a portion of input 302 to semantic context and memory repository 306, which can perform a semantic search on asset 308 and / or other knowledge contained in semantic context and memory repository 306. The semantic search can generate a list of candidate actions that can be used to resolve input 302 or any subset thereof from all actions that can be performed via one or more assets 308. In some embodiments, candidate actions can be generated based solely on context information. For example, input 302 can be compared with metadata of actions to generate candidate actions. Table 2 lists specific examples of context information and candidate actions that can be received by routing engine 304 and execution plans that can be generated by routing engine 304.

[0095] Table 2:

[0096]

[0097]

[0098]

[0099]

[0100] As a specific example, routing engine 304 receives the context information and candidate actions listed in Table 2 as part of a prompt transmitted to routing engine 304. Input 302 may be an instruction from the user to create an expense, or it may be a continuation of a conversation listed along with the conversation history in Table 2. Each candidate action has an associated agent, description, JSON schema, and action type. The JSON schema may contain input argument slots with specified required or optional types. As an example, the "Get Expense Details" action has an associated JSON schema with an input argument slot for the expense ID, which is an integer and required. Some actions, such as "Get Expense Categories," have associated schemas without any input argument slots.

[0101] Routing engine 304 can use candidate actions to form input cues for a generative AI model. This generative AI model can be or is included in generative AI model 310, which may include one or more large language models (LLMs). Routing engine 304 can be communicatively coupled to generative AI model 310 via a common language model interface layer (CLMI layer 312). CLMI layer 312 can be an adapter layer that allows routing engine 304 to invoke various different generative AI models that can be included in generative AI model 310. For example, routing engine 304 can generate input cues and provide them to CLMI layer 312, which can transform the input cues into model-specific input cues for input to a specific generative AI model. Routing engine 304 can receive output from this specific generative AI model that can be used to generate an execution plan. This output can be or includes an execution plan. In other embodiments, the output can be used by routing engine 304 as input to allow routing engine 304 to generate an execution plan. Based on the utterance contained in input 302, the output may include a list containing one or more executable actions. In some embodiments, the execution plan may include an ordered list of actions to be performed to resolve input 302.

[0102] In some instances, routing engine 304 can perform slot filling to supplement any information required by execution engine 314 to execute the execution plan. In some examples, the output of routing engine 304 to be sent to execution engine 314 can be in JSON schema format. The output can have an associated schema with specified key-value pairs required by execution engine 314, and routing engine 304 can determine if any information required for the selected action is missing. Routing engine 304 can use conversation history, text from input 302, context, or any combination thereof to determine the missing information. For example, an action might require information related to the current date, and routing engine 304 can retrieve the current data from input 302 or from information available within the context. Routing engine 304 can customize actions for users by recognizing user preferences and filling input argument slots within the schema according to those preferences.

[0103] As a specific example, routing engine 304 can select the "Create Expense" action from the candidate actions listed in Table 2 using the context information and input 302 listed in Table 2. "Create Expense" is associated with an API call as its action type and with a JSON pattern containing input arguments, including a required integer employee ID, a required floating-point expense amount, a required date formatted as a date type, a required string for the merchant name, and an optional string describing the location of the expense. Routing engine 304 retrieves the employee ID from user profile data retrieved along with the context information and can determine the merchant by reviewing the conversation history and recognizing that the user previously stated "the merchant is Burger King." Routing engine 304 may be unable to determine the remaining input argument slots and instead sets their values ​​to null. Routing engine 304 generates an execution plan as listed in Table 2, including the action, agent, and arguments for the pattern.

[0104] The routing engine 304 can transmit an execution plan to the execution engine 314 for execution. The routing engine 304 can transmit the execution plan along with any information required by the execution engine 314. The execution engine 314 can perform iterative processing on each executable action contained in the execution plan. For example, for each executable action, the execution engine 314 can identify the action type, invoke one or more states for executing that action type, and use assets to execute the executable action to obtain output. The execution engine 314 can be communicatively coupled to an action executor 316 that can be configured to perform at least a portion of the iterative processing. For example, the action executor 316 can identify one or more action types for each executable action contained in the execution plan. In a particular example, the action executor 316 can identify a first action type 318a for a first executable action of the execution plan. The first action type 318a can be or contain a semantic action such as summary text, or other suitable semantic actions. Additionally or alternatively, the action executor 316 can identify a second action type 318b for a second executable action of the execution plan. The second action type 318b may involve invoking an API, such as an API for adjusting an account or other suitable API. Additionally or alternatively, the action executor 316 may identify a third action type 318c for a third executable action of the execution plan. The third action type 318c may be or include knowledge actions, such as providing an answer to a technical question or other suitable knowledge actions. In some embodiments, the third action type 318c may involve invoking at least one generative artificial intelligence model in generative artificial intelligence model 310 to retrieve specific knowledge or a specific answer. In other embodiments, the third action type 318c may involve invoking semantic context and memory repository 306 or other knowledge documents.

[0105] In some instances, execution engine 314 may not receive all the information required by action executor 316 to execute the requested action. Execution engine 314 can instead generate an execution failure status, which can then be sent to response engine 320. As a specific example, execution engine 314 can receive the execution plan listed in Table 2. The arguments for amount, date, and location for the "Create Cost" action in the execution plan are set to null. Amount and date are required arguments depending on the JSON schema associated with the "Create Cost" action. Calling an API without required arguments may fail, and execution engine 314 may indicate the missing arguments in the execution status, or attempt to call the API and generate the execution status based on the API call's output.

[0106] Action executor 316 can continue iterative processing based on the action type indicated by the executable actions contained in the execution plan. Once action executor 316 identifies the action type, it can identify and / or invoke one or more states for each executable action based on the action type. The state of an action can involve indications about whether the action can be executed or has already been executed. For example, the state of a particular executable action can include "preparing," "ready," "executing," "successful," "failed," or any other suitable state. Based on the state of the invoked executable action, action executor 316 can determine whether the executable action is ready to be executed, and if the executable action is not yet ready to be executed, action executor 316 can identify any missing information or assets required to continue executing the executable action. In response to determining that the executable action is ready to be executed, and in response to determining that no dependencies exist for the executable action (or that existing dependencies are satisfied), action executor 316 can execute the executable action to generate output.

[0107] Action executor 316 can execute each executable action, or any subset thereof, contained in the execution plan to generate a set of outputs. This set of outputs may include knowledge outputs, semantic outputs, API outputs, and other suitable outputs. Action executor 316 can provide this set of outputs to output engine 320. Output engine 320 can be configured to generate a second input cue based on this set of outputs. The second input cue can be provided to at least one generative AI model in generative AI model 310 to generate a response 322 to input 302. Output engine 320 can invoke at least one generative AI model to generate the response 322, which can be provided to the user in response to input 302.

[0108] In some instances, the response engine 320 may receive execution failure status from the execution engine 314. The execution failure status may contain information about why the execution engine 314 was unable to complete the action. The response engine 320 may generate a response 322 to the user, instructing the agent that additional information is needed to complete the action. The response 322 may request the user to input the required information. In some examples, the response 322 may indicate that the action could not be completed and provide an explanation of why. In some examples, the response 322 may be sent to the user before the action is executed, and the user may be asked to confirm details of the execution plan. As a specific example, the response engine 320 may request a fee amount and date from the user to complete the "Create Fee" action as listed in Table 2. Upon receiving the fee amount request from the user, the routing engine 304 may update the execution plan.

[0109] In some embodiments, at least one generative artificial intelligence model for generating response 322 may be similar to or the same as at least one generative artificial intelligence model for generating output to generate an execution plan, or in other cases, the same model.

[0110] Figure 4 This is a simplified block diagram illustrating a data flow for populating associated patterns of actions using a routing engine, according to various embodiments, for a digital assistant capable of executing an execution plan in response to a user's utterance. Figure 4 As shown, routing engine 402 can implement different solutions for routing utterances to execution engine 406. First data stream 400a illustrates an example of routing utterances to the execution engine by performing slot filling and transmitting the selected action and parameters. Second data stream 400b illustrates an example of routing utterances to execution engine 406 by sending the selected action without performing slot filling.

[0111] As shown in the first data stream 400a, the routing engine 402 may receive an input query 404, which may contain a set of parameters and contextual information. In some examples, the input query 404 may be a rich text utterance from a user. The routing engine 402 may retrieve a list 410 of candidate actions and a context 412 from a semantic context and a memory repository 408. The candidate actions 410 may be retrieved from a data repository 414. The context 412 may contain contextual information about the user (e.g., profile information, employee ID) and user preferences related to a specific action or goal. The context 412 may also contain a conversation history 416 between the digital assistant and the user and one or more historical execution plans 418. The historical execution plan 418 may contain one or more previously executed actions and the results of those actions. The routing engine 402 may be an LLM capable of selecting actions, or may use an LLM to select an appropriate action in response to the input query 404. The routing engine 402 may select actions based on information determined from the input query 404. In some examples, input query 404 may not provide enough information for routing engine 402 to select action 420. The routing engine can use contextual information retrieved from input query 404, context 412, conversation history 416, historical execution plan 418, or any combination thereof to select the appropriate action.

[0112] Action 420 may have an associated schema that requires one or more argument slots. Additionally or alternatively, these one or more argument slots may include optional argument slots. As an example, an action may be an API call with an associated JSON schema, which may have required and / or optional key-value pairs. Routing engine 402 may identify or determine one or more parameters 422 corresponding to one or more missing argument slots. The process of determining parameters 422 and assigning them to the missing arguments in the schema associated with action 420 may be referred to as slot filling. In some instances, parameters 422 may be determined from input query 404. In some instances, one or more parameters 422 may not be determined from input query 404. Routing engine may determine one or more parameters 422 from input query 404, context 412, historical execution plans, conversation history, or any combination thereof. In some examples, input query 404 may refer to previous parts of a conversation between the user and the digital assistant, and routing engine 402 may search conversation history 416 to find information related to the relevant parameters 422 of input query 404 and / or action 420.

[0113] In some examples, action 420 may be an API-based action and may require a JSON schema with specific arguments to execute. As an example, input query 404 could be a request to change a user's 401k payment. Routing engine 402 could choose to invoke the 401k Change Payment API as action 420, and the 401k Change Payment API might require the payment amount as an argument slot. Routing engine 402 can search for available context information from input query 404 or context 412 to identify the payment amount. In some instances, a user may have previously mentioned a payment amount and subsequently requested a change to it. In this instance, routing engine 402 could search for the previously mentioned payment amount in conversation history 416 and enter that amount accordingly. As a specific example, routing engine 402 selects the "Create Fee" action detailed in Table 2 and uses the context information listed in Table 2 to determine the arguments for the associated JSON schema (e.g., employee ID, fee amount, fee date, merchant, location).

[0114] In some examples, action 420 can be a rich text action. The routing engine 402 can generate JSON with a single actual argument for the query and use the context information required by the response input query 404 to generate the corresponding query.

[0115] In some instances, parameter 422 needs to belong to a specific version, and routing engine 402 can adjust the type of parameter 422 to match the pattern. For example, the 401k Change Payment API might require the payment amount to be a floating-point number, but routing engine 402 might retrieve the payment amount as a string. Routing engine 402 can correct the data type of the payment amount from string to floating-point before inputting the argument. In some instances, routing engine 402 might not be able to correct the version of parameter 422 to match the pattern. Routing engine 402 can search the input query 404, context 412, conversation history 416, historical execution plan 418, or any combination thereof to retrieve the correct version of one or more parameters 422. As a specific example, the "Create Fee" action listed in Table 2 requires the "Fee Amount" argument to be a floating-point number, but routing engine 402 might receive the fee amount as a string and correct the type to floating-point before populating the "Fee Amount" argument slot.

[0116] In some examples, routing engine 402 may fail to retrieve parameter 422, or may fail to retrieve the correct version of parameter 422. Routing engine 402 may leave missing argument slots empty in the schema. In some examples, the corresponding parameter 422 may be set to empty. If a missing argument slot in the schema is optional, action 420 can be executed even with the missing argument slot. As a specific example, routing engine 402 may select the "Create Fee" action from the candidate actions listed in Table 2. Routing engine 402 retrieves the employee ID from the user profile information and the merchant from the conversation history listed along with the context information in Table 2. Routing engine 402 may fail to determine the amount, date, and location and set the corresponding input argument slots in the associated JSON schema to empty. The amount and date are required arguments, and if execution fails, the user may be requested further information.

[0117] After slot filling is complete, routing engine 402 can transmit an execution plan containing action 420 and parameters 422 corresponding to the actual slots in the associated mode to execution engine 406. Execution engine 406 can execute the transmitted action 420 and generate output 424. Execution engine 406 can update context 412 by adding the execution plan and output 424 to historical execution plan 418, and can transmit output 424 to response engine 426. After successful execution of action 420, output 424 can indicate that action 420 has been successfully executed. Response engine 426 can generate a response for the user indicating that action 420 has been successfully completed.

[0118] In some examples, routing engine 402 can route to agent 428 before filling the slot for the selected action execution. Routing engine 402 can retrieve one or more candidate agents from semantic context and memory repository 408 via semantic search. Routing engine 402 can select the appropriate agent 428 based on input query 404 and context 412. Agent 428 can be associated with a pool of actions that can be executed by agent 428. Routing engine 402 or a second LLM-based action routing model can select action 420 from the action pool based on input query 404, context 412, conversation history 416, historical execution plan 418, or any combination thereof. After selecting action 420, routing engine 402 or the action routing model can determine one or more parameters 422 and perform slot filling as described above. Action 420 and parameters 422 can then be transmitted to execution engine 406 as part of the execution plan.

[0119] In some instances, action 420 may not be executed successfully. Output 424 can indicate that action 420 was not executed successfully and the reason for the failure. In some instances, action 420 may not be executed successfully due to the absence of one or more required parameters 422. Response engine 426 can generate a response to the user indicating that more information may be needed to execute action 420 and request one or more missing parameters 422 from the user. In some instances, action 420 may not be executed successfully because one or more parameters 422 are incorrect versions. Response engine 426 can generate a response to the user requesting the correct version of one or more parameters 422. Response engine 426 can update context 412 by adding the response to the user to conversation history 416. Upon receiving a subsequent response from the user with missing parameters or corrected versions of parameters, routing engine 402 can populate the corresponding slot in the associated pattern of action 420 and transmit the updated parameters 422 to execution engine 406.

[0120] As shown in the second data stream 400b, the routing engine 402 may not perform any slot filling before transmitting the selected action 428 to the execution engine 406. The routing engine 402 may retrieve a set of candidate actions 410 from the semantic context and memory repository 408. In some examples, the input query 404 and / or one or more most recent queries may be used to retrieve candidate actions 410. In some examples, one or more candidate agents may be retrieved from the semantic context and memory repository 408, and candidate actions 410 may be selected from a set of actions associated with those one or more candidate agents. The routing engine 402 may use limited contextual information sufficient for action planning and may determine the selected action 430 based on the input query 404 and the context 412.

[0121] In some examples, candidate action 410 and selected action 430 may not have associated patterns, and routing engine 402 may determine that slot filling is irrelevant to the routing decision. In some examples, execution engine 406 can be configured to perform slot filling as required by the API-based action, as described above. In some examples, execution engine 406 can be configured to collect missing slot values. Upon receiving selected action 430, execution engine 406 can determine that one or more input argument slots are missing in the associated pattern. Instead of attempting to execute selected action 430, execution engine 406 can generate output 424 indicating which parameters are missing and / or the versions of the required parameters. Response engine 426 can generate a response to the user indicating that the action was not executed due to missing information and requesting the missing information from the user. Missing parameters can be collected within a limited number of requests to the user, and the failure of selected action 430 to execute due to missing parameters can be prevented.

[0122] Flowchart for enriching the application context with responses

[0123] Figure 5 This is a flowchart of a process 500 that uses a routing engine to populate input arguments within an associated pattern in response to an input query to generate an execution plan, according to various embodiments. Figure 5 The processing described herein can be implemented using software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores), hardware, or a combination thereof of the corresponding system. The software can be stored on a non-transitory storage medium (e.g., a memory device). Figure 5 The processes presented and described below are intended to be illustrative and non-limiting. Although Figure 5 The illustrations depict various processing steps that occur in a specific order or sequence, but this is not intended to be limiting. In some alternative embodiments, these steps may be performed in a different order, or some steps may be performed at least partially in parallel. In some embodiments, Figure 5 The processing described in the text can be determined by... Figure 1-4 One or more of the components, computing devices, services, etc., illustrated and described are used to perform actions such as digital assistants, generative artificial intelligence models (LLMs), etc.

[0124] At step 505, an input query is received from the user. The input query may include specific data. In some examples, receiving the input query may include receiving contextual information. The contextual information may include conversation history and historical execution plans associated with the user. The input query may originate from the user's natural language utterance.

[0125] At step 510, an action is identified from one or more candidate actions based on the input query. In some examples, the action is identified based on the input query, conversation history, and historical execution plans. In some examples, a generative artificial intelligence model is used to identify the action to be performed from the candidate actions based on the input query, conversation history, and historical execution plans.

[0126] At step 515, a set of input parameter slots within a pattern associated with the action is identified. In some examples, a first subset of the input parameter slots is determined, wherein the first subset includes the input parameter slots required to perform the action. In some examples, a second subset of the input parameter slots is determined, wherein the second subset includes input parameter slots optional to perform the action.

[0127] At step 520, each input argument slot in the set of input parameters is filled. For each input argument slot, process 500 includes determining whether one or more parameters corresponding to that input argument slot can be derived from specific data. Based on the one or more parameters corresponding to that input argument slot, the one or more parameters can be derived from the specific data, and the input argument slot can be filled with versions of one or more parameters that conform to the pattern.

[0128] In some examples, it is determined that at least one input argument slot in the set cannot be filled with one or more parameters, where one or more parameters may be missing at least one parameter. The missing parameter can be extracted from the conversation history using a generative AI model, and the input argument slot can be filled with that at least one parameter.

[0129] In some examples, it is determined that at least one input parameter in the set of input argument slots cannot be filled using one or more parameters, where the version of one or more parameters does not conform to the pattern. The version of one or more parameters can be adjusted to conform to the pattern, and the adjusted version of one or more parameters in the pattern can be used to fill the at least one input argument slot.

[0130] In some examples, process 500 may determine that the first subset includes at least one input argument slot that cannot be filled with a version of the one or more parameters. Process 500 may determine whether context information contained in the input query includes one or more indications of whether a version of the one or more parameters is included. Based on the determination that the context information includes a version of the one or more parameters, the version of the one or more parameters is used to fill the at least one input argument slot. Based on the determination that the context information does not include a version of the one or more parameters, output is generated that can be used to request subsequent input from the user to receive a version of the one or more parameters.

[0131] In some examples, process 500 may determine that the second subset includes at least one input argument slot that cannot be populated with a version of the one or more parameters. Process 500 may determine whether the context information contained in the input query includes one or more indications of whether a version of the one or more parameters is included. Based on the determination that the context information includes a version of the one or more parameters, the version of the one or more parameters is used to populate the at least one input argument slot. Based on the determination that the context information does not include a version of the one or more parameters, the group of populated input argument slots is filled with empty slots for the at least one input argument slot. The group of populated input argument slots may be transferred to the execution engine.

[0132] At step 525, an execution plan containing the actions with the set of populated input argument slots is passed to the execution engine. The execution engine can be configured to execute actions to generate a response to the input query. In some examples, the execution plan is executed using the set of populated input argument slots and the execution engine to generate a response to the input query. This response can be passed to the user to facilitate interaction involving that user.

[0133] Examples of architectures used to implement cloud infrastructure

[0134] As noted above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also offer various services to accompany these infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, and clustering software, etc.). Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0135] In some instances, IaaS customers can access resources and services over a wide area network (WAN) such as the internet, and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0136] In most cases, cloud computing models will require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0137] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). This is typically managed by the cloud provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for disposition (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0138] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0139] In some cases, IaaS provisioning presents two distinct challenges. First, there's the initial challenge of provisioning the initial infrastructure set before anything is operational. Second, once everything is provisioned, there's the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.). In some cases, both challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some instances, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0140] In some examples, the infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more inbound / outbound traffic group rules may also be provided to define how inbound / outbound traffic to the network and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., may also be provided. The infrastructure can evolve incrementally as more and / or more infrastructure elements are expected and added.

[0141] In some instances, continuous deployment techniques can be used to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they expect to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, the infrastructure on which the code will be deployed must first be set up. In some instances, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, the code can be deployed using deployment tools.

[0142] Figure 6 This is a block diagram 600 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service operator 602 may be communicatively coupled to a secure host lease 604, which may include a virtual cloud network (VCN) 606 and a secure host subnet 608. In some examples, service operator 602 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software (such as Microsoft Windows Mobile®) and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc.), and supporting the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. The client computing device can be a workstation computer running a variety of commercially available UNIX® or UNIX-like operating systems, including but not limited to any of the various GNU / Linux operating systems (such as, for example, Google Chrome OS). Alternatively or additionally, the client computing device can be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect® gesture input), and / or a personal messaging device capable of communicating over a network that can access VCN 606 and / or the internet.

[0143] VCN 606 may include a local peering gateway (LPG) 610, which may be communicatively coupled to a secure shell (SSH) VCN 612 via an LPG 610 included in an SSH VCN 612. SSH VCN 612 may include an SSH subnet 614, and SSH VCN 612 may be communicatively coupled to a control plane VCN 616 via an LPG 610 included in a control plane VCN 616. Furthermore, SSH VCN 612 may be communicatively coupled to a data plane VCN 618 via an LPG 610. Control plane VCN 616 and data plane VCN 618 may be contained within a service lease 619 that may be owned and / or operated by an IaaS provider.

[0144] The control plane VCN 616 may include a control plane demilitarized zone (DMZ) layer 620 that acts as a peripheral network (e.g., a portion of a corporate network between an intranet and an external network). DMZ-based servers can assume limited liability and help control vulnerabilities. Furthermore, the DMZ layer 620 may include one or more load balancer (LB) subnets 622, a control plane application layer 624 that may include one or more application subnets 626, and a control plane data layer 628 that may include one or more database (DB) subnets 630 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 622 contained in the control plane DMZ layer 620 may be communicatively coupled to one or more application subnets 626 contained in the control plane application layer 624 and an Internet gateway 634 that may be contained in the control plane VCN 616. The application subnets 626 may be communicatively coupled to one or more DB subnets 630 contained in the control plane data layer 628, as well as a service gateway 636 and a Network Address Translation (NAT) gateway 638. The control plane VCN 616 may include the service gateway 636 and the NAT gateway 638.

[0145] The control plane VCN 616 may include a data plane mirror application layer 640, which may include one or more application subnets 626. The one or more application subnets 626 included in the data plane mirror application layer 640 may include a virtual network interface controller (VNIC) 642 capable of executing a compute instance 644. The compute instance 644 may communicatively couple the one or more application subnets 626 of the data plane mirror application layer 640 to the one or more application subnets 626 that may be included in the data plane application layer 646.

[0146] Data plane VCN 618 may include data plane application layer 646, data plane DMZ layer 648, and data plane data layer 650. Data plane DMZ layer 648 may include one or more LB subnets 622 communicatively coupled to one or more application subnets 626 of data plane application layer 646 and Internet gateway 634 of data plane VCN 618. One or more application subnets 626 may be communicatively coupled to service gateway 636 and NAT gateway 638 of data plane VCN 618. Data plane data layer 650 may also include one or more DB subnets 630 communicatively coupled to one or more application subnets 626 of data plane application layer 646.

[0147] The Internet gateway 634 of the control plane VCN 616 and data plane VCN 618 can be communicatively coupled to the metadata management service 652, which in turn can be communicatively coupled to the public Internet 654. The public Internet 654 can be communicatively coupled to the NAT gateway 638 of the control plane VCN 616 and data plane VCN 618. The service gateway 636 of the control plane VCN 616 and data plane VCN 618 can be communicatively coupled to the cloud service 656.

[0148] In some examples, the service gateway 636 of the control plane VCN 616 or data plane VCN 618 can make application programming interface (API) calls to the cloud service 656 without traversing the public internet 654. API calls from the service gateway 636 to the cloud service 656 can be unidirectional: the service gateway 636 can make API calls to the cloud service 656, and the cloud service 656 can send requested data to the service gateway 636. However, the cloud service 656 may not initiate API calls to the service gateway 636.

[0149] In some examples, secure host lease 604 can be directly connected to service lease 619, which would otherwise be isolated. Secure host subnet 608 can communicate with SSH subnet 614 via LPG 610, which enables bidirectional communication between otherwise isolated systems. Connecting secure host subnet 608 to SSH subnet 614 allows secure host subnet 608 to access other entities within service lease 619.

[0150] Control plane VCN 616 allows users of service lease 619 to configure or otherwise provision desired resources. Desired resources provisioned in control plane VCN 616 can be deployed or otherwise used in data plane VCN 618. In some examples, control plane VCN 616 can be isolated from data plane VCN 618, and the data plane mirror application layer 640 of control plane VCN 616 can communicate with the data plane application layer 646 of data plane VCN 618 via VNIC 642, which can be included in both the data plane mirror application layer 640 and the data plane application layer 646.

[0151] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 654, which can transmit requests to the metadata management service 652. The metadata management service 652 can transmit the request to the control plane VCN 616 via internet gateway 634. The request can be received by one or more LB subnets 622 contained in the control plane DMZ layer 620. The LB subnets 622 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 626 contained in the control plane application layer 624. If the request is validated and requires a call to the public internet 654, the call to the public internet 654 can be transmitted to a NAT gateway 638 that can make calls to the public internet 654. The request may expect the stored metadata to be stored in one or more DB subnets 630.

[0152] In some examples, the data plane mirroring application layer 640 can facilitate direct communication between the control plane VCN 616 and the data plane VCN 618. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 618. Through VNIC 642, the control plane VCN 616 can communicate directly with the resources contained in the data plane VCN 618, and thus can perform configuration changes, updates, or other appropriate modifications.

[0153] In some embodiments, the control plane VCN 616 and data plane VCN 618 may be included in a service lease 619. In this case, the system's users or customers may not own or operate the control plane VCN 616 or data plane VCN 618. Alternatively, the IaaS provider may own or operate both the control plane VCN 616 and data plane VCN 618, and both planes may be included in the service lease 619. This embodiment can enable the isolation of networks that might prevent users or customers from interacting with resources from other users or customers. Furthermore, this embodiment can allow users or customers of the system to privately store databases without relying on the public Internet 654, which may not have the desired level of threat prevention for storage.

[0154] In other embodiments, one or more LB subnets 622 included in the control plane VCN 616 may be configured to receive signals from the service gateway 636. In this embodiment, the control plane VCN 616 and the data plane VCN 618 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 654. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on a service lease 619, which may be isolated from the public internet 654.

[0155] Figure 7 This is a block diagram 700 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 702 (e.g., Figure 6 The service provider (602) can communicatively couple to the secure host lease (704) (e.g., Figure 6 Secure hosting lease 604), the secure hosting lease 704 may include a Virtual Cloud Network (VCN) 706 (e.g., Figure 6 VCN606) and Secure Host Subnet 708 (e.g., Figure 6 The secure host subnet 608). VCN 706 may include a local peering gateway (LPG) 710 (e.g., Figure 6 The LPG 610), which can be communicatively coupled to the Secure Shell (SSH) VCN 712 (e.g., via the LPG 610 contained in the SSH VCN 712) Figure 6 SSH VCN 612). SSH VCN 712 can include SSH subnet 714 (e.g., Figure 6 SSH subnet 614), and SSH VCN 712 can be communicatively coupled to control plane VCN 716 via LPG 710 included in control plane VCN 716 (e.g., Figure 6Control plane VCN 616). Control plane VCN 716 may be included in service lease 719 (e.g., Figure 6 In the service lease 619), and the data plane VCN 718 (e.g., Figure 6 The data plane VCN 618 may be included in a customer lease 721 that may be owned or operated by a user or customer of the system.

[0156] Control plane VCN 716 may include control plane DMZ layer 720 (e.g., Figure 6 The control plane DMZ layer 620), which may include one or more LB subnets 722 (e.g., Figure 6 (one or more) LB subnets 622), may include (one or more) application subnets 726 (e.g., Figure 6 The control plane application layer 724 of (one or more) application subnets 626 (e.g., Figure 6 The control plane application layer 624) may include one or more database (DB) subnets 730 (e.g., similar to...). Figure 6 The control plane data layer 728 of (one or more) DB subnets 630 (e.g., Figure 6 The control plane data layer 628). One or more LB subnets 722 contained in the control plane DMZ layer 720 can be communicatively coupled to one or more application subnets 726 contained in the control plane application layer 724 and an Internet gateway 734 that can be contained in the control plane VCN 716 (e.g., Figure 6 Internet gateway 634), and application subnet(s) 726 can communicatively couple to DB subnet(s) 730 contained in control plane data layer 728 and service gateway 736 (e.g., Figure 6 Service gateway 636) and Network Address Translation (NAT) gateway 738 (e.g., Figure 6 (NAT gateway 638). The control plane VCN 716 may include the service gateway 736 and the NAT gateway 738.

[0157] The control plane VCN 716 may include a data plane mirror of the application layer 740, which may include one or more application subnets 726 (e.g., Figure 6 The data plane mirror application layer 740). One or more application subnets 726 contained in the data plane mirror application layer 740 may include compute instances 744 capable of performing computations (e.g., similar to...). Figure 6The virtual network interface controller (VNIC) 742 (e.g., the VNIC of 642) of the computing instance 644. The computing instance 744 may facilitate the mirroring of the application layer 740 with one or more application subnets 726 and may be included in the data plane application layer 746 (e.g., Figure 6 Communication between one or more application subnets 726 in the data plane application layer 646 via VNIC 742 contained in the data plane mirror application layer 740 and VNIC 742 contained in the data plane application layer 746.

[0158] The Internet gateway 734 included in the control plane VCN 716 can be communicatively coupled to the metadata management service 752 (e.g., Figure 6 Metadata management service 752), which can communicatively couple to the public Internet 754 (e.g., Figure 6 The public internet 754 can communicatively couple to a NAT gateway 738 contained in the control plane VCN 716. The service gateway 736 contained in the control plane VCN 716 can communicatively couple to a cloud service 756 (e.g., ...). Figure 6 Cloud services 656).

[0159] In some examples, data plane VCN 718 may be included in customer lease 721. In this case, the IaaS provider may provide control plane VCN 716 for each customer, and the IaaS provider may set up a unique compute instance 744 for each customer, included in service lease 719. Each compute instance 744 may allow communication between control plane VCN 716 included in service lease 719 and data plane VCN 718 included in customer lease 721. Compute instance 744 may allow resources provisioned in control plane VCN 716 included in service lease 719 to be deployed or otherwise used in data plane VCN 718 included in customer lease 721.

[0160] In other examples, an IaaS provider's customer may have a database residing in customer lease 721. In this example, control plane VCN 716 may include a data plane mirror application layer 740, which may include one or more application subnets 726. Data plane mirror application layer 740 may reside in data plane VCN 718, but it may not reside in data plane VCN 718. That is, data plane mirror application layer 740 may have access to customer lease 721, but it may not reside in data plane VCN 718 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 740 may be configured to invoke data plane VCN 718, but it may not be configured to invoke any entity contained in control plane VCN 716. Customers may expect to deploy or otherwise use resources provisioned in the control plane VCN 716 in the data plane VCN 718, and the data plane mirroring application layer 740 can facilitate the customer's desired deployment or other use of resources.

[0161] In some embodiments, an IaaS provider's customer can apply filters to data plane VCN 718. In this embodiment, the customer can determine what data plane VCN 718 can access, and the customer can restrict access from data plane VCN 718 to the public Internet 754. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 718's access to any external networks or databases. Applying filters and controls to data plane VCN 718 contained in customer lease 721 helps isolate data plane VCN 718 from other customers and the public Internet 754.

[0162] In some embodiments, cloud service 756 may be invoked by service gateway 736 to access services that may not exist on public internet 754, control plane VCN 716, or data plane VCN 718. The connection between cloud service 756 and control plane VCN 716 or data plane VCN 718 may not be real-time or continuous. Cloud service 756 may reside on different networks owned or operated by an IaaS provider. Cloud service 756 may be configured to receive calls from service gateway 736 and may be configured not to receive calls from public internet 754. Some cloud services 756 may be isolated from other cloud services 756, and control plane VCN 716 may be isolated from cloud services 756 that may not be in the same region as control plane VCN 716. For example, control plane VCN 716 may be located in "Region 1," and cloud service "Deployment 6" may be located in both "Region 1" and "Region 2." If the service gateway 736, contained in the control plane VCN 716 located in region 1, makes a call to deployment 6, then that call can be transmitted to deployment 6 in region 1. In this example, the control plane VCN 716 or deployment 6 in region 1 may not be communicatively coupled to or otherwise communicate with deployment 6 in region 2.

[0163] Figure 8 This is a block diagram 800 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 802 (e.g., Figure 6 The service provider (602) can communicatively couple to the secure host lease (804) (e.g., Figure 6 Secure hosting lease 604), the secure hosting lease 804 may include a Virtual Cloud Network (VCN) 806 (e.g., Figure 6 VCN606) and Secure Host Subnet 808 (e.g., Figure 6 The secure host subnet 608). VCN 806 can include LPG 810 (e.g., Figure 6 The LPG 610), which can be communicatively coupled to the SSH VCN 812 via the LPG 810 included in the SSH VCN 812 (e.g., Figure 6 SSH VCN 612). SSH VCN 812 can include SSH subnet 814 (e.g., Figure 6 SSH subnet 614), and SSH VCN 812 can be communicatively coupled to control plane VCN 816 via LPG 810 included in control plane VCN 816 (e.g., Figure 6 The control plane VCN 616) and coupled to the data plane VCN 818 via the LPG 810 contained in the data plane VCN 818 (e.g., Figure 6Data plane 618). Control plane VCN 816 and data plane VCN 818 can be included in service lease 819 (e.g., Figure 6 In the service rental (619).

[0164] The control plane VCN 816 may include one or more load balancer (LB) subnets 822 (e.g., Figure 6 The control plane DMZ layer 820 of (one or more) LB subnets 622) (e.g., Figure 6 The control plane DMZ layer 620 may include one or more application subnets 826 (e.g., similar to...). Figure 6 The control plane application layer 824 of (one or more) application subnets 626 (e.g., Figure 6 The control plane application layer 624), may include (one or more) DB subnets 830, and the control plane data layer 828 (e.g., Figure 6 The control plane data layer 628). One or more LB subnets 822 contained in the control plane DMZ layer 820 can be communicatively coupled to one or more application subnets 826 contained in the control plane application layer 824 and an Internet gateway 834 that can be contained in the control plane VCN 816 (e.g., Figure 6 Internet gateway 634), and application subnet 826 can communicatively couple to DB subnet 830 contained in control plane data layer 828 and service gateway 836 (e.g., Figure 6 The service gateway) and the Network Address Translation (NAT) gateway 838 (e.g., Figure 6 (NAT gateway 638). The control plane VCN 816 may include the service gateway 836 and the NAT gateway 838.

[0165] The data plane VCN 818 may include the data plane application layer 846 (e.g., Figure 6 Data plane application layer 646), data plane DMZ layer 848 (e.g., Figure 6 Data plane DMZ layer 648), and data plane data layer 850 (e.g., Figure 6The data plane data layer 850. The data plane DMZ layer 848 may include one or more trusted application subnets 860 and one or more untrusted application subnets 862 communicatively coupled to the data plane application layer 846, and one or more LB subnets 822 of the Internet gateway 834 contained in the data plane VCN 818. The one or more trusted application subnets 860 may be communicatively coupled to the service gateway 836 contained in the data plane VCN 818, the NAT gateway 838 contained in the data plane VCN 818, and one or more DB subnets 830 contained in the data plane data layer 850. The one or more untrusted application subnets 862 may be communicatively coupled to the service gateway 836 contained in the data plane VCN 818 and the one or more DB subnets 830 contained in the data plane data layer 850. The data plane data layer 850 may include one or more DB subnets 830 communicatively coupled to the service gateway 836 contained in the data plane VCN 818.

[0166] One or more untrusted application subnets 862 may include one or more primary VNICs 864(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 866(1)-(N). Each tenant VM 866(1)-(N) may be communicatively coupled to a corresponding application subnet 867(1)-(N) that may be contained in a corresponding container egress VCN 868(1)-(N), which may be contained in a corresponding customer lease 870(1)-(N). A corresponding secondary VNIC 872(1)-(N) may facilitate communication between one or more untrusted application subnets 862 contained in a data plane VCN 818 and the application subnets contained in the container egress VCN 868(1)-(N). Each container egress VCN 868(1)-(N) may include a NAT gateway 838 that can be communicatively coupled to the public Internet 854 (e.g., Figure 6 Public Internet 654).

[0167] An Internet gateway 834, contained in the control plane VCN 816 and the data plane VCN 818, can be communicatively coupled to a metadata management service 852 (e.g., Figure 6A metadata management system 652, which is communicatively coupled to the public internet 854, is also communicatively coupled to a NAT gateway 838 contained in a control plane VCN 816 and a data plane VCN 818. A service gateway 836 contained in both the control plane VCN 816 and the data plane VCN 818 is communicatively coupled to a cloud service 856.

[0168] In some embodiments, the data plane VCN 818 can be integrated with the customer lease 870. Such integration may be useful or desired by the IaaS provider's customers in certain situations, such as when support may be expected during code execution. Customers may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider can determine whether to run the code provided by the customer to the IaaS provider.

[0169] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality attached to the data plane application layer 846. The code running this functionality may execute in VMs 866(1)-(N) and may not be configured to run anywhere else on the data plane VCN 818. Each VM 866(1)-(N) may be connected to a customer lease 870. The corresponding container 871(1)-(N) contained in VMs 866(1)-(N) may be configured to run the code. In this case, dual isolation may exist (e.g., container 871(1)-(N) runs the code, where container 871(1)-(N) may be contained in at least one VM 866(1)-(N) contained in untrusted application subnet 862), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 871(1)-(N) may be communicatively coupled to customer lease 870 and may be configured to transmit or receive data from customer lease 870. Containers 871(1)-(N) may not be configured to transmit or receive data from any other entity in the data plane VCN 818. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of containers 871(1)-(N).

[0170] In some embodiments, one or more trusted application subnets 860 may run code that may be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 860 may be communicatively coupled to one or more database subnets 830 and configured to perform CRUD operations in one or more database subnets 830. One or more untrusted application subnets 862 may be communicatively coupled to one or more database subnets 830, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 830. Containers 871(1)-(N) that may be contained in each customer's VM 866(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 830.

[0171] In other embodiments, the control plane VCN 816 and the data plane VCN 818 may be coupled without direct communication. In this embodiment, there may be no direct communication between the control plane VCN 816 and the data plane VCN 818. However, communication can occur indirectly through at least one method. The LPG 810 may be established by an IaaS provider, which can facilitate communication between the control plane VCN 816 and the data plane VCN 818. In another example, either the control plane VCN 816 or the data plane VCN 818 may invoke the cloud service 856 via the service gateway 836. For example, an invocation of the cloud service 856 from the control plane VCN 816 may include a request for a service that can communicate with the data plane VCN 818.

[0172] Figure 9 This is a block diagram 900 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 902 (e.g., Figure 6 The service provider (602) can communicatively couple to the secure host lease (904) (e.g., Figure 6 Secure hosting lease 604), the secure hosting lease 904 may include a Virtual Cloud Network (VCN) 906 (e.g., Figure 6 VCN606) and Secure Host Subnet 908 (e.g., Figure 6 The secure host subnet 608). VCN 906 can include LPG 910 (e.g., Figure 6 The LPG 610), the LPG 910 can be accessed via SSH VCN 912 (e.g., LPG 610), Figure 6 The LPG 910 in SSH VCN 912 is communicatively coupled to SSH VCN 912. SSH VCN 912 may include SSH subnet 914 (e.g., Figure 6SSH subnet 614), and SSH VCN 912 can be communicatively coupled to control plane VCN 916 via LPG 910 contained in control plane VCN 916 (e.g., Figure 6 The control plane VCN 616) and coupled to the data plane VCN 918 via the LPG910 contained in the data plane VCN 918 (e.g., Figure 6 Data plane 618). Control plane VCN 916 and data plane VCN 918 can be included in service lease 919 (e.g., Figure 6 In the service rental (619).

[0173] The control plane VCN 916 may include one or more LB subnets 922 (e.g., Figure 6 The control plane DMZ layer 920 of (one or more) LB subnets 622) (e.g., Figure 6 The control plane DMZ layer 620), may include (one or more) application subnets 926 (e.g., Figure 6 The control plane application layer 924 of (one or more) application subnets 626 (e.g., Figure 6 The control plane application layer 624) may include (one or more) DB subnets 930 (e.g., Figure 8 The control plane data layer 928 of (one or more) DB subnets 830 (e.g., Figure 6 The control plane data layer 628). One or more LB subnets 922 contained in the control plane DMZ layer 920 can be communicatively coupled to one or more application subnets 926 contained in the control plane application layer 924 and an Internet gateway 934 that can be contained in the control plane VCN 916 (e.g., Figure 6 Internet gateway 634), and application subnet(s) 926 can communicatively couple to DB subnet(s) 930 contained in control plane data layer 928 and service gateway 936 (e.g., Figure 6 The service gateway) and Network Address Translation (NAT) gateway 938 (e.g., Figure 6 (NAT gateway 638). The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0174] The data plane VCN 918 may include the data plane application layer 946 (e.g., Figure 6 Data plane application layer 646), data plane DMZ layer 948 (e.g., Figure 6 Data plane DMZ layer 648), and data plane data layer 950 (e.g., Figure 6The data plane data layer 650). The data plane DMZ layer 948 may include one or more trusted application subnets 960 that can be communicatively coupled to the data plane application layer 946 (e.g., Figure 8 (one or more) trusted application subnets 860) and (one or more) untrusted application subnets 962 (e.g., Figure 8 The data plane VCN 918 includes one or more untrusted application subnets 862 and one or more LB subnets 922 of Internet gateway 934. One or more trusted application subnets 960 may communicatively couple to a service gateway 936, a NAT gateway 938, and one or more DB subnets 930 in the data plane VCN 918. One or more untrusted application subnets 962 may communicatively couple to a service gateway 936 and a DB subnet 930 in the data plane VCN 918. The data plane VCN 950 may include one or more DB subnets 930 that may communicatively couple to a service gateway 936 in the data plane VCN 918.

[0175] One or more untrusted application subnets 962 may include a primary VNIC 964(1)-(N) communicatively coupled to tenant virtual machines (VMs) 966(1)-(N) residing within one or more untrusted application subnets 962. Each tenant VM 966(1)-(N) may run code in a corresponding container 967(1)-(N) and is communicatively coupled to an application subnet 926 that may be contained in a data plane application layer 946 contained in a container egress VCN 968. A corresponding secondary VNIC 972(1)-(N) may facilitate communication between one or more untrusted application subnets 962 contained in a data plane VCN 918 and the application subnets contained in a container egress VCN 968. The container egress VCN may include a public internet 954 (e.g., Figure 6 The public internet (654) uses NAT gateway 938.

[0176] The Internet gateway 934, contained in the control plane VCN 916 and the data plane VCN 918, can be communicatively coupled to the metadata management service 952 (e.g., Figure 6A metadata management system 952 (database management service 952) can be communicatively coupled to the public internet 954. The public internet 954 can be communicatively coupled to a NAT gateway 938 contained in a control plane VCN 916 and a data plane VCN 918. A service gateway 936 contained in a control plane VCN 916 and a data plane VCN 918 can be communicatively coupled to a cloud service 956.

[0177] In some examples, Figure 9 The architecture shown in the block diagram 900 can be considered as... Figure 8 This is an exception to the pattern shown in the architecture of block diagram 800, and this pattern may be what the IaaS provider's customers would expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The customer has real-time access to the corresponding container 967(1)-(N) contained in each customer's VM 966(1)-(N). Container 967(1)-(N) can be configured to invoke the corresponding auxiliary VNIC 972(1)-(N) contained in one or more application subnets 926 of the data plane application layer 946, which may be contained in the container egress VCN 968. The auxiliary VNIC 972(1)-(N) can transmit the invocation to a NAT gateway 938, which can transmit the invocation to the public internet 954. In this example, the container 967(1)-(N) that can be accessed by the customer in real-time can be isolated from the control plane VCN 916 and from other entities contained in the data plane VCN 918. Container 967(1)-(N) can also be isolated from resources from other customers.

[0178] In other examples, a client can use container 967(1)-(N) to invoke cloud service 956. In this example, the client can run code within container 967(1)-(N) requesting a service from cloud service 956. Container 967(1)-(N) can then transmit the request to auxiliary VNIC 972(1)-(N), which can then transmit the request to a NAT gateway, which can then transmit the request to public internet 954. Public internet 954 can then transmit the request via internet gateway 934 to one or more LB subnets 922 contained in control plane VCN 916. In response to determining that the request is valid, one or more LB subnets can then transmit the request to one or more application subnets 926, which can then transmit the request to cloud service 956 via service gateway 936.

[0179] It should be recognized that the IaaS architectures 600, 700, 800, and 900 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or component arrangements.

[0180] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0181] Figure 10 An example computer system 1000 in which various embodiments can be implemented is illustrated. System 1000 can be used to implement any of the computer systems described above. As shown, computer system 1000 includes a processing unit 1004 that communicates with a plurality of peripheral subsystems via a bus subsystem 1002. These peripheral subsystems may include a processing acceleration unit 1006, an I / O subsystem 1008, a storage subsystem 1018, and a communication subsystem 1024. Storage subsystem 1018 includes a tangible computer-readable storage medium 1022 and system memory 1010.

[0182] Bus subsystem 1002 provides a mechanism for allowing various components and subsystems of computer system 1000 to communicate with each other as intended. While bus subsystem 1002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1002 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Microchannel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0183] A processing unit 1004, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1000. One or more processors may be included in the processing unit 1004. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1004 may be implemented as one or more independent processing units 1032 and / or 1034, each including a single-core or multi-core processor. In other embodiments, the processing unit 1004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0184] In various embodiments, processing unit 1004 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 1004 and / or storage subsystem 1018. With appropriate programming, processor(s) 1004 can provide the various functions described above. Computer system 1000 may additionally include processing acceleration unit 1006, which may include digital signal processor (DSP), dedicated processor, etc.

[0185] I / O subsystem 1008 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices such as the Microsoft Xbox® 360 game controller via a natural user interface using gestures and voice commands. User interface input devices may also include eye posture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., “blinking” when taking a photo and / or making menu selections) and translates the eye posture into input to an input device (e.g., Google Glass®). Furthermore, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0186] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0187] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1000 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0188] Computer system 1000 may include a storage subsystem 1018 that provides a tangible, non-transitory, computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc., which, when executed by one or more cores or processors of processing unit 1004, provide the aforementioned functionality. Storage subsystem 1018 may also provide a repository for storing data used according to this disclosure.

[0189] As in Figure 10 As illustrated in the example, storage subsystem 1018 may include various components, including system memory 1010, computer-readable storage medium 1022, and computer-readable storage medium reader 1020. System memory 1010 may store program instructions that can be loaded and executed by processing unit 1004. System memory 1010 may also store data used during instruction execution and / or data generated during program instruction execution. Various types of programs may be loaded into system memory 1010, including but not limited to client applications, web browsers, middleware applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0190] System memory 1010 may also store operating system 1016. Examples of operating system 1016 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercial UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS and Palm® OS. In some embodiments where computer system 1000 executes one or more virtual machines, the virtual machine, along with its guest operating system (GOS), may be loaded into system memory 1010 and executed by one or more processors or cores of processing unit 1004.

[0191] Depending on the type of computer system 1000, system memory 1010 can have different configurations. For example, system memory 1010 can be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Different types of RAM configurations can be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some implementations, system memory 1010 may include a basic input / output system (BIOS), which contains basic routines that facilitate, for example, the transfer of information between elements within computer system 1000 during startup.

[0192] The computer-readable storage medium 1022 may represent a remote, local, fixed and / or removable storage device, as well as a storage medium for temporarily and / or more permanently containing and storing computer-readable information (including instructions executable by the processing unit 1004 of the computer system 1000) for use by the computer system 1000.

[0193] Computer-readable storage medium 1022 may include any suitable medium known or used in the art, including storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassette, magnetic tape, disk storage or other magnetic storage devices, or other tangible computer-readable media.

[0194] For example, computer-readable storage medium 1022 may include hard disk drives that read from or write to non-removable non-volatile magnetic media, disk drives that read from or write to removable non-volatile magnetic disks, and optical disc drives that read from or write to removable non-volatile optical discs (such as CD ROMs, DVDs, and Blu-ray® discs or other optical media). Computer-readable storage medium 1022 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital audio tapes, and so on. Computer-readable storage medium 1022 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules and other data for computer system 1000.

[0195] Machine-readable instructions executable by one or more processors or cores of processing unit 1004 may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile memory devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard disk drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0196] The communication subsystem 1024 provides interfaces to other computer systems and networks. The communication subsystem 1024 serves as an interface for receiving data from other systems and sending data from computer system 1000 to other systems. For example, the communication subsystem 1024 enables computer system 1000 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1024 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technology, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 1024 may provide a wired network connection (e.g., Ethernet).

[0197] In some embodiments, the communication subsystem 1024 may also represent one or more users who can use the computer system 1000 to receive input communications in the form of structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc.

[0198] For example, the communication subsystem 1024 can be configured to receive data feeds 1026 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0199] Furthermore, the communication subsystem 1024 can also be configured to receive data in the form of a continuous data stream, which may include an event stream 1028 and / or event updates 1030 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0200] The communication subsystem 1024 can also be configured to output structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1000.

[0201] The computer system 1000 can be one of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google® Glass head-mounted displays), PCs, workstations, mainframes, information stations, server racks, or any other data processing system.

[0202] Due to the constantly evolving nature of computers and networks, the description of the computer system 1000 depicted in the figures is merely a concrete example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may be used and / or specific elements may be implemented using hardware, firmware, software (including applets), or combinations thereof. Additionally, connections to other computing devices, such as network input / output devices, may also be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0203] While specific embodiments have been described, various modifications, alterations, alternative constructions, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within certain specific data processing environments, but can freely operate within multiple data processing environments. Furthermore, although the embodiments have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the series of transactions and steps described. The various features and aspects of the above embodiments can be used individually or in combination.

[0204] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using only hardware, only software, or a combination thereof. The various processes described herein can be implemented in any combination on the same processor or on different processors. Accordingly, where a component or service is described as being configured to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuitry to perform the operations, by programming programmable electronic circuitry (such as a microprocessor), or any combination thereof. Processes can communicate using various technologies, including but not limited to conventional inter-process communication technologies, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0205] The embodiments may be implemented using a computer program product comprising a computer program / instruction that, when executed by a processor, causes the processor to perform any of the methods described in the disclosure.

[0206] Accordingly, the specification and drawings are to be considered illustrative rather than restrictive. However, it will be apparent that additions, omissions, deletions, and other modifications and alterations may be made thereto without departing from the broader spirit and scope set forth in the claims. Therefore, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0207] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar designations, are to be interpreted as covering both singular and plural, unless otherwise indicated herein or clearly contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” are to be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connected” should be interpreted as partially or wholly contained in, attached to, or joined together, even if something exists in between. Unless otherwise indicated herein, the enumeration of value ranges herein is intended only as a shorthand method for individually referencing each individual value falling within that range, and each individual value is incorporated into the specification as if it were individually enumerated herein. Unless otherwise indicated herein or clearly contradicted by the context, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate the embodiments and does not constitute a limitation on the scope of this disclosure, unless otherwise stated. Nothing in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0208] Disjunctive language, such as the phrase “at least one of X, Y, or Z”, unless otherwise explicitly stated, is intended to be understood in the context generally used to represent items, terms, etc., and may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive language is generally not intended to, and should not, imply that some embodiments require the presence of at least one of X, at least one of Y, or at least one of Z, each individually.

[0209] This document describes preferred embodiments of the present disclosure, including the best modes known for carrying out the present disclosure. Variations of those preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. Those skilled in the art should be able to suitably employ such variations and may practice the present disclosure in ways other than those specifically described herein. Accordingly, the present disclosure includes all modifications and equivalents to the subject matter recited in the appended claims, where permitted by applicable law. Furthermore, unless otherwise indicated herein, the present disclosure includes any combination of the foregoing elements in all its possible variations.

[0210] The embodiments may be implemented using a computer program product comprising a computer program / instruction that, when executed by a processor, causes the processor to perform any of the methods described in the disclosure.

[0211] All references cited in this article, including publications, patent applications and patents, are incorporated into this article by reference to the same extent as if each reference individually and specifically indicated to be incorporated by reference and elaborated in full in this article.

[0212] In the foregoing specification, various aspects of this disclosure have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A computer-implemented method, comprising: Receive input queries from users, the input queries including specific data; Based on the input query, identify an action from one or more candidate actions; Identify a set of input parameter slots within a pattern associated with the action; For each input parameter slot in the set of input parameter slots, the input parameter slot is filled in the following manner: Determine whether one or more parameters corresponding to the input parameter slot can be derived from the specific data, and Based on the one or more parameters corresponding to the input argument slot, (i) derive the one or more parameters from the specific data, and (ii) populate the input argument slot with a version of the one or more parameters that conforms to the pattern; as well as The execution plan, which includes the set of input argument slots that have been filled, is transmitted to the execution engine configured to execute the action to generate a response to the input query.

2. The computer-implemented method of claim 1, wherein receiving the input query further includes receiving context information, wherein the context information includes (i) a conversation history associated with the user and (ii) a historical execution plan, and wherein identifying the action includes identifying the action based on the input query, the conversation history and the historical execution plan.

3. The computer-implemented method of claim 2, wherein identifying the action further includes using a generative artificial intelligence model to select the action to be performed from the candidate actions based on the input query, the conversation history, and the historical execution plan.

4. The computer-implemented method according to any one of claims 1 to 3, further comprising: It is determined that at least one input parameter slot in the set of input parameter slots cannot be filled with the one or more parameters, wherein the one or more parameters are missing at least one parameter; The generative artificial intelligence model is used to extract at least one parameter from the conversation history; as well as Use the at least one parameter to fill the at least one input parameter slot.

5. The computer-implemented method according to any one of claims 1 to 3, further comprising: It is determined that at least one input argument slot in the set of input argument slots cannot be filled with the one or more parameters, wherein the version of the one or more parameters does not conform to the pattern; Adjust the version of one or more parameters to conform to the pattern; as well as In the mode, the adjusted versions of the one or more parameters are used to fill the at least one input argument slot.

6. The computer-implemented method according to any one of claims 1 to 5, further comprising: Determine a first subset of the set of input parameter slots, wherein the first subset includes the input parameter slots required to perform the action; as well as Determine a second subset of the set of input parameter slots, wherein the second subset includes input parameter slots that are optional for performing the action.

7. The computer-implemented method as described in claim 6, further comprising: Based on determining that the first subset includes at least one input argument slot that cannot be filled with the version of the one or more parameters, determine whether the context information contained in the input query includes one or more indications of the version of the one or more parameters; Based on one or more indications that determine the context information includes the version of the one or more parameters, the version of the one or more parameters is used to populate the at least one input argument slot; as well as Based on the one or more indications that determine the context information does not include the version of the one or more parameters, an output is generated that can be used to request subsequent input from the user to receive the version of the one or more parameters.

8. The computer-implemented method as described in claim 6, further comprising: Based on determining that the second subset includes at least one input argument slot that cannot be filled with the version of the one or more parameters, determine whether the context information contained in the input query includes one or more indications of the version of the one or more parameters; Based on one or more indications that determine the context information includes the version of the one or more parameters, the version of the one or more parameters is used to populate the at least one input argument slot; as well as Based on the one or more indications that determine the context information does not include the version of the one or more parameters, the already filled set of input parameter slots are filled with empty slots for the at least one input parameter slot, and the already filled set of input parameter slots are transmitted to the execution engine.

9. The computer-implemented method according to any one of claims 1 to 9, further comprising: The execution plan is executed using the execution engine and the pre-filled set of input argument slots to generate a response to the input query; as well as The response is transmitted to the user to facilitate interaction involving the user.

10. A system comprising: One or more processors; as well as One or more computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations, including: Receive input queries from users, the input queries including specific data; Based on the input query, identify an action from one or more candidate actions; Identify a set of input parameter slots within a pattern associated with the action; For each input parameter slot in the set of input parameter slots, the input parameter slot is filled in the following manner: Determine whether one or more parameters corresponding to the input parameter slot can be derived from the specific data, and Based on the one or more parameters corresponding to the input argument slots, (i) derive the one or more parameters from the specific data, and (ii) populate the input argument slots with a pattern-compliant version of the one or more parameters; and The execution plan, which includes the set of input argument slots that have been filled, is transmitted to the execution engine configured to execute the action to generate a response to the input query.

11. The system of claim 10, wherein receiving the input query further includes receiving context information, wherein the context information includes (i) a conversation history associated with the user and (ii) a historical execution plan, and wherein identifying the action includes identifying the action based on the input query, the conversation history and the historical execution plan.

12. The system of claim 11, wherein the operation of identifying the action further includes using a generative artificial intelligence model to select the action to be performed from the candidate actions based on the input query, the conversation history, and the historical execution plan, and wherein the operation further includes: It is determined that at least one input parameter slot in the set of input parameter slots cannot be filled with the one or more parameters, wherein the one or more parameters are missing at least one parameter; The generative artificial intelligence model is used to extract at least one parameter from the conversation history; as well as Use the at least one parameter to fill the at least one input parameter slot.

13. The system of any one of claims 10 to 12, wherein the operation further comprises: It is determined that at least one input argument slot in the set of input argument slots cannot be filled with the one or more parameters, wherein the version of the one or more parameters does not conform to the pattern; Adjust the version of one or more parameters to conform to the pattern; as well as In the mode, the adjusted versions of the one or more parameters are used to fill the at least one input argument slot.

14. The system of any one of claims 10 to 13, wherein the operation further comprises: Determine a first subset of the set of input parameter slots, wherein the first subset includes the input parameter slots required to perform the action; as well as Determine a second subset of the set of input parameter slots, wherein the second subset includes input parameter slots that are optional for performing the action.

15. The system of claim 14, wherein the operation further comprises: Based on determining that the first subset includes at least one input argument slot that cannot be filled with the version of the one or more parameters, determine whether the context information contained in the input query includes one or more indications of the version of the one or more parameters; Based on one or more indications that determine the context information includes the version of the one or more parameters, the version of the one or more parameters is used to populate the at least one input argument slot; as well as Based on the one or more indications that determine the context information does not include the version of the one or more parameters, an output is generated that can be used to request subsequent input from the user to receive the version of the one or more parameters.

16. The system of claim 14, wherein the operation further comprises: Based on determining that the second subset includes at least one input argument slot that cannot be filled with the version of the one or more parameters, determine whether the context information contained in the input query includes one or more indications of the version of the one or more parameters; Based on one or more indications that determine the context information includes the version of the one or more parameters, the version of the one or more parameters is used to populate the at least one input argument slot; as well as Based on the one or more indications that determine the context information does not include the version of the one or more parameters, the already filled set of input parameter slots are filled with empty slots for the at least one input parameter slot, and the already filled set of input parameter slots are transmitted to the execution engine.

17. The system of any one of claims 10 to 16, wherein the operation further comprises: The execution plan is executed using the execution engine and the pre-filled set of input argument slots to generate a response to the input query; as well as The response is transmitted to the user to facilitate interaction involving the user.

18. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations, including: Receive input queries from users, the input queries including specific data; Based on the input query, identify an action from one or more candidate actions; Identify a set of input parameter slots within a pattern associated with the action; For each input parameter slot in the set of input parameter slots, the input parameter slot is filled in the following manner: Determine whether one or more parameters corresponding to the input parameter slot can be derived from the specific data, and Based on the one or more parameters corresponding to the input argument slot, (i) derive the one or more parameters from the specific data, and (ii) populate the input argument slot with a version of the one or more parameters that conforms to the pattern; as well as The execution plan, which includes the set of input argument slots that have been filled, is transmitted to the execution engine configured to execute the action to generate a response to the input query.

19. The one or more non-transitory computer-readable media of claim 18, wherein receiving the input query further includes receiving context information, wherein the context information includes (i) a conversation history associated with the user and (ii) a historical execution plan, and wherein identifying the action includes identifying the action based on the input query, the conversation history and the historical execution plan.

20. One or more non-transitory computer-readable media as claimed in claim 18 or 19, wherein the operation of identifying the action further comprises selecting the action to be performed from the candidate actions using a generative artificial intelligence model based on the input query, the conversation history, and the historical execution plan, and wherein the operation further comprises: It is determined that at least one input parameter slot in the set of input parameter slots cannot be filled with the one or more parameters, wherein the one or more parameters are missing at least one parameter; The generative artificial intelligence model is used to extract at least one parameter from the conversation history; as well as Use the at least one parameter to fill the at least one input parameter slot.

21. An apparatus comprising components for implementing the computer-implemented method as described in any one of claims 1 to 9.

22. A computer program product comprising computer instructions that, when executed by a processor, implement the operation of the computer-implemented method as described in any one of claims 1 to 9.

Images