Enterprise-generated artificial intelligence architecture

Abstract

Systems and methods manage a plurality of agents with an orchestrator to generate a response to an input. The orchestrator employs one or more multi-modal models, such as a large language model, to process or deconstruct prompts into a series of instructions for different agents. Each agent employs one or more machine learning models to process different inputs or different portions of inputs associated with the cues. The system generates a natural language summary of structured and unstructured data records with an orchestrator. The system formulates a natural language summary for outputting and transmitting data records.

Description

Technical Field

[0001] The present disclosure relates to generative artificial intelligence and machine learning. More specifically, the present disclosure relates to an enterprise generative artificial intelligence architecture. Background Art

[0002] Artificial intelligence (AI) is a branch of computer science used to develop software that enables computer systems to perform tasks that mimic human cognitive intelligence, such as visual perception, speech recognition, decision making, and language translation. Traditional methods for storing and retrieving information typically involve databases and applications for indexing and locating specific files for searches. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 A diagram depicting an example logical flow of an enterprise generative artificial intelligence system, according to some embodiments.

[0004] Figure 2 A diagram depicting an example layered architecture and environment for an enterprise generative artificial intelligence system, according to some embodiments.

[0005] Figure 3 A diagram depicting an example architecture for an enterprise generative artificial intelligence system, according to some embodiments.

[0006] Figure 4 A diagram depicts an example network system for enterprise generative artificial intelligence, according to some embodiments.

[0007] Figure 5 A diagram depicts an example enterprise generative artificial intelligence system, in accordance with some embodiments.

[0008] Figure 6 Flowchart depicting an example generative AI unstructured data and structured data retrieval process.

[0009] Figure 7 A diagram depicting an example logical flow of an enterprise generative artificial intelligence system, according to some embodiments.

[0010] Figure 8A Depicted is a flow diagram of an example iterative generative artificial intelligence process using unstructured data, according to some embodiments.

[0011] Figures 8B to 8C Depicted is a flow diagram of an example non-iterative generative artificial intelligence process using unstructured data, in accordance with some embodiments.

[0012] Figure 9 Depicted is a flow diagram of an example iterative generative artificial intelligence process using unstructured data, according to some embodiments.

[0013] Figure 10 Depicted is a flow diagram of an example generative artificial intelligence process using unstructured and structured data, according to some embodiments.

[0014] Figure 11 Depicted is a flow diagram of an example generative artificial intelligence process using unstructured and structured data, according to some embodiments.

[0015] Figure 12 A flow diagram depicts an example of a non-iterative generative artificial intelligence process using unstructured data, according to some embodiments.

[0016] Figure 13 A flow diagram depicts an example of a generative artificial intelligence process using structured data, according to some embodiments.

[0017] Figure 14 A flowchart depicts example operation of an enterprise generative artificial intelligence system according to some embodiments.

[0018] Figure 15 is a diagram of an example computer system for implementing features disclosed herein, according to some embodiments. DETAILED DESCRIPTION

[0019] Generative AI is an artificial intelligence technology that uses machine learning algorithms to mimic human cognitive intelligence and generate content. This content can take the form of text, audio, video, images, and more. Content in enterprise computing environments is typically distributed across disparate, potentially incompatible, siloed, and access-controlled data sources. Supporting efficient search capabilities is further complicated by the need for subject matter expertise or context-specific knowledge.

[0020] This paper discloses an architecture for enterprise generative AI to transform the interaction with enterprise information that fundamentally changes the human-computer interaction (HCI) model for enterprise software. Enterprises running sensitive workloads in cloud-native, on-premises, or air-gapped environments can implement an enterprise generative AI architecture to generate enterprise-wide insights in response to simple intuitive inputs, using intelligent agents that develop and orchestrate complex operations, using tools for rapid location and retrieval. The enterprise generative AI architecture enables enterprise users to ask open-ended, multi-level, context-specific questions that are processed using generative AI with machine learning to understand the request, identify relevant information, and generate new context-specific insights using predictive analytics. The enterprise generative AI architecture supports simplified human-computer interaction with an intuitive natural language interface and advanced accessibility features for adaptable forms of input (including but not limited to text, audio, video, images, etc.).

[0021] Conventional generative artificial intelligence processes are computationally inefficient, often contain erroneous or biased information, cannot effectively handle different types of inputs and outputs, cannot effectively utilize different data sources with different data formats, and cannot effectively interact with other machine learning systems or effectively utilize information across different domains. These problems and those discussed above are addressed by the enterprise generative artificial intelligence systems and processes discussed in this article. More specifically, enterprise generative artificial intelligence systems can efficiently provide more accurate and reliable results than conventional generative artificial intelligence solutions, while consuming less computing resources and requiring shorter processing time. In addition, enterprise generative artificial intelligence systems can adopt various models that effectively provide cross-domain functionality. Enterprise generative artificial intelligence systems can also use a combination of agents and tools to efficiently process various inputs received from different data sources (e.g., with different data formats) and return results in a common data format (e.g., natural language).

[0022] The enterprise generative AI architecture includes an orchestrator agent (or simply orchestrator) that oversees, controls, and / or otherwise manages a number of different agents and tools. The orchestrator may include one or more machine learning models and may perform supervisory functions, such as routing inputs (e.g., queries, instruction sets, natural language input or other human-readable input, machine-readable input) to specific agents to complete a specified set of tasks (e.g., retrieval requests specified by the orchestrator to answer queries). The machine learning models may include some or all of the different types or modalities of models described herein (e.g., multimodal machine learning models, large language models, data models, statistical models, audio models, vision models, audio-visual models, etc.). Agents may include one or more multimodal models (e.g., large language models) to use a variety of different tools to complete specified tasks. Different agents may use various tools to execute and process unstructured data retrieval requests, structured data retrieval requests, and API calls (e.g., for accessing AI application insights). A tool may include one or more specific functions and / or machine learning models to complete a given task (or set of tasks).

[0023] The agent can be adapted to perform in different ways based on the context. The context can relate to a specific domain (e.g., an industry), and the agent can employ specific models (e.g., large language models, other machine learning models, and / or data models) that have been trained on industry-specific datasets (such as healthcare datasets, etc.). A specific agent can use a healthcare model when receiving input associated with a healthcare environment, and can also be easily and efficiently adapted to use different models based on different inputs or contexts. In fact, some or all of the models described herein can be trained for specific domains in addition to or instead of more general purposes. The enterprise generative AI architecture leverages domain-specific models to produce accurate context-specific retrieval and insights.

[0024] The orchestrator manages agents to efficiently process different inputs or different parts of inputs. For example, the inputs may require the system to access and retrieve data records from different data sources (e.g., unstructured data stores, structured data stores, and time series data stores), database tables from different types of databases, and machine learning insights from different machine learning applications. Different agents can handle each of these requests independently and in parallel, greatly improving computational efficiency.

[0025] Agents can process different data returned by different agents and / or tools. For example, large language models typically receive input in natural language format. Agents can receive information in non-natural language format from tools (e.g., database tables, images, audio) and transform it into natural language that describes the tool's output in a format understood by the large language model. The large language model can then process this input to "answer" or otherwise satisfy the initial input.

[0026] Figure 1 A diagram 100 depicts an example logic flow of an enterprise generative artificial intelligence system according to some embodiments. As shown, initial input 102 is received by the system from a user (e.g., natural language input) or another system (e.g., machine-readable input).

[0027] The orchestrator agent (or simply orchestrator) may pre-process the input in step 104. Pre-processing may include, for example, acronym handling, translation handling, punctuation handling, input recognition (e.g., identifying different parts of the input 102 to be processed by different agents). The orchestrator may further process the input 102 using a multimodal model (e.g., a large language model) to create a plan for determining the outcome of the input (step 112). The plan may include a set of prescribed tasks, such as a structured data retrieval task, an unstructured data retrieval task, a time series processing task, and a visualization task. In some embodiments, the plan may specify which tools 108 should be used to perform the tasks, and the orchestrator may select an agent based on the specified tools. In some embodiments, the plan may specify which agents should be used to perform the tasks, and the agents may independently specify which tools 108 should be used to perform the tasks.

[0028] continue Figure 1 For example, the orchestrator routes the pre-processed input to an agent 106 for further processing. More specifically, the orchestrator may use one or more multimodal models (e.g., language, video, audio, statistical models, etc.) and / or other machine learning models to interpret the input 102 to select an appropriate agent 106 and an appropriate tool 108. For example, the orchestrator may determine that a first portion of the input requires a database query, while another portion of the input requires an API call. The orchestrator may appropriately route the first portion of the input to an appropriate agent 106-1 (e.g., a structured data retrieval agent) and the second portion of the input to another agent 106-2 (e.g., an API agent). There may be any number of such agents 106 accessing any number of different tools 108. The orchestrator may also instruct the agents 106 to operate in parallel and / or serially.

[0029] The agent 106 can select an appropriate tool 108 to complete a set of specified tasks (e.g., tasks specified by the orchestrator). The tool 108 can make appropriate function calls to retrieve different data records and other functions. As used herein, data records can include unstructured data records (e.g., documents and text data stored on a file system in formats such as PDF, DOCX, .MD, HTML, TXT, PPTX, image files, audio files, video files, and application output), structured data records (e.g., database tables or other data records stored according to a data model or type system), time series data records (e.g., sensor data, artificial intelligence application insights), and / or other types of data records (e.g., access control lists). The agent 106 can transform different data records into a common format (e.g., a natural language format) that can be post-processed (step 110) by a large language model (e.g., the same or different large language model that performed pre-processing). More specifically, post-processing can obtain tool output (and / or transformed tool output) and generate a final result that satisfies the initial input (step 112). For example, the orchestrator can use one or more large language models to determine the result. If the orchestrator determines that there is not enough information to satisfy the initial input, the orchestrator may iteratively repeat some or all of the above steps until a stopping condition is met and / or there is enough information to generate a final result (step 112).

[0030] Figure 2 Diagram 200 depicts an example layered architecture and environment for an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. Figure 2 In the example of FIG, the enterprise generative artificial intelligence system architecture and environment includes a hierarchy of layers. More specifically, the hierarchy of layers includes an input layer 202, a supervision layer 210, an agent layer 220, an agent and tool layer 230, a tool and data model layer 250, and an external layer 280. It will be understood that these layers are shown by way of example, and other examples may include any number of such layers (e.g., any number of layers 220 and 230).

[0031] Input layer 202 represents the layer of the enterprise generative artificial intelligence system architecture that receives input (e.g., queries, complex inputs, and / or instruction sets, etc.) from a user or system. For example, an interface module of the enterprise generative artificial intelligence system can receive input.

[0032] The supervisory layer 210 represents the following layer of the enterprise generative artificial intelligence system architecture, which includes one or more large language models (e.g., of an orchestrator module) that can develop plans for responding to input received in the input layer 202. The plans can include a set of specified tasks (e.g., retrieval tasks and API call tasks, etc.). In one example, the supervisory layer 210 can provide the pre-processing and post-processing functionality described herein, as well as the functionality of the orchestrator and understanding modules described herein. The supervisory layer 210 can coordinate with one or more of the subsequent layers 220-280 to perform the specified set of tasks.

[0033] The agent layer 220 represents the layer of the enterprise generative artificial intelligence system architecture that includes agents that can perform a set of specified tasks. Figure 2 In the example of , the agent layer 220 includes a machine learning insight agent 222, an information retrieval agent 224, a dashboard agent 226, and an optimizer agent 228. Each of the agents 224-228 can include a large language model that provides reasoning functionality for completing its assigned portion of a specified set of tasks. More specifically, the agents 224-228 can instruct agents and tools in any number of subsequent layers (e.g., layer 230) that may exist to perform tasks. For example, the machine learning insight agent 222 can instruct the text processing tool 232 to perform a text processing task (e.g., convert the output of an artificial intelligence application into natural language), instruct the image processing tool 234 to perform an image processing task (e.g., generate a natural language summary of an image output from an artificial intelligence application), instruct the time series tool 236 to obtain summarized time series data (e.g., time series data output from an artificial intelligence application), and instruct the API tool 238 to perform an API call task (e.g., execute an API call to trigger or access an artificial intelligence application).

[0034] The information retrieval agent 224 can collaborate and / or coordinate with several different agents to perform retrieval tasks. For example, the information retrieval agent 224 can instruct the unstructured data retriever agent 240 to receive unstructured data records, instruct the structured data retriever agent 242 to retrieve structured data records, and instruct the type system retriever agent 244 to obtain one or more data models (or subsets of data models) and / or types from the type system. The type system provides compatibility across different data formats, protocols, operating languages, different systems, etc. A type can encapsulate some or all of the different types or modalities described herein (e.g., multimodal, textual, encoded, linguistic, statistical, audio, visual, audiovisual, etc.). For example, a data model can include a variety of different types (e.g., in a tree or graph structure), and each type can describe data fields, operations, functions, etc. Each type can represent a different object (e.g., a real-world object, such as a machine or sensor in a factory, etc.) or system (e.g., a computing cluster, an enterprise data store, a file system), and each type can include a large language model context that provides context for the large language model to design or update plans. For example, the context can include a natural language summary or description of the type (e.g., a description of the object represented, relationships to other types or objects, and associated methods and functions, etc.). Types can be defined in a natural language format for efficient processing by the large language model. The type system retriever agent 244 can traverse the data model 254 to retrieve subsets of the data model 254 and / or types of the data model 254. The structured data retriever agent 242 can then use the retrieved information to efficiently retrieve structured data from a structured data source (e.g., a structured data source structured or modeled according to the data model 254).

[0035] The dashboard agent 226 may be configured to generate one or more visualizations and / or graphical user interfaces, such as dashboards, etc. For example, the dashboard agent 226 may execute tools 252-5 and 252-6 to generate a dashboard based on information retrieved by other agents and / or information output by other agents (e.g., natural language summaries of associated tool outputs).

[0036] The optimizer agent 228 can be configured to perform various prescriptive analytical functions and mathematical optimizations 252-7 to assist in computing answers to various questions. For example, the large language model 206 can use the optimizer agent 228 to generate a plan, determine a set of prescriptive tasks, and determine whether more information is needed to generate a final result.

[0037] Tools and data model layer 250 is intended to represent a layer of the enterprise generative AI system architecture that includes tools 252 and data models 254. Agents 240-242 can execute tools 252 to retrieve information from various applications and data stores 282 in external layer 280 (e.g., external to the enterprise generative AI system). Tools 252 can include connectors that can connect to systems and data stores external to the enterprise generative AI system.

[0038] Figure 3 Diagram 300 depicts an example architecture of an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. Figure 3 In the example of FIG, an enterprise generative AI system can ingest different data, such as unstructured data 302, structured data (e.g., tables) 304, sensor data 306, and access control information 308. The data can be received via one or more AI data pipelines 310. The data can be ingested according to an object model (or data model) 312, and an embedding model 314 (e.g., a ColBERT implementation) can be used to generate embeddings from the ingested data and persisted and / or virtualized in various data stores 318. The data stores 318 can include vector data stores (e.g., a FAISS implementation), metadata data stores, virtualized data stores, distributed file systems, key-value data stores, and feature stores (e.g., storing embeddings as features of the various models described herein). A database engine and time series engine 316 can also be used to persist and / or virtualize the data within the data stores 318.

[0039] exist Figure 3 In the example of FIG, the enterprise generative artificial intelligence system includes various different agents 326-339. These are shown by way of example, and various embodiments may include different agents instead of or in addition to agents 326-339. Figure 5 Further details regarding the agents and other features of the enterprise generative artificial intelligence system are found in the other figures presented herein.

[0040] exist Figure 3In the example of FIG, an enterprise generative artificial intelligence system includes an orchestrator 342 with a fine-tuned large language model. In some embodiments, orchestrator 342 and / or agents 326-339 may include and / or access task-specific large language models 348-356, as well as external or third-party large language models 340. Orchestrator 342 may utilize various underlying platform service tools, such as runtime hardware profiles 368, end-to-end retraining 370, logging and monitoring 372, hint registry 374, model registry 376, hosted JUPYTER environment 378, and / or access management controls 380.

[0041] In some embodiments, user query 362 and / or other input can be received by an application hosting application engine 360, which can communicate with low-latency engine 358 to provide input or transformed input to orchestrator 342. Orchestrator 342 can leverage various agents, large language models, and other features to generate accurate and reliable (e.g., free of hallucinations) answers to user query 362.

[0042] In some embodiments, Figure 3 Only a portion of the architecture depicted in

[15] may be deployed in an external environment (e.g., a customer-hosted environment or a customer-cloud environment). For example, a portion of the architecture may be deployed in an external environment while some or all of the other portions remain in an internal environment (e.g., an internally hosted environment and / or associated cloud environment of an entity providing an enterprise generative artificial intelligence system).

[0043] Figure 4 A diagram 400 depicts an example network system for enterprise generative artificial intelligence, according to some embodiments. Figure 4 In the example, the network system includes enterprise generative artificial intelligence system 402, enterprise systems 404-1 through 404-N (individually enterprise system 404 and collectively enterprise systems 404), external systems 406-1 through 406-N (individually external system 406 and collectively external systems 406), and communication network 408.

[0044] Enterprise generative artificial intelligence system 402 can function to iteratively and non-iteratively generate machine learning model inputs and outputs to determine a final output (e.g., an "answer" or "result") in response to initial input (e.g., provided by a user or another system). In some embodiments, the functionality of enterprise generative artificial intelligence system 402 can be performed by one or more servers (e.g., cloud-based servers) and / or other computing devices. Enterprise generative artificial intelligence system 402 can be implemented using a type system and / or a model-driven architecture.

[0045] In various implementations, the enterprise generative artificial intelligence system 402 can provide a variety of different technical features, such as efficiently processing and generating complex natural language input and output, generating synthetic data (e.g., to supplement customer data obtained during the onboarding process or otherwise fill in data gaps), generating source code (e.g., application development), generating applications (e.g., artificial intelligence applications), providing cross-domain functionality, and countless other technical features not provided by traditional systems. As used herein, synthetic data can refer to content generated on the fly (e.g., by a large language model) as part of the processes described herein. Synthetic data can also include ephemeral content that is not retrieved (e.g., temporary data that does not exist in the database), as well as a combination of retrieved information, queried information, and / or model output.

[0046] In some embodiments, the enterprise generative artificial intelligence system 402 can provide and / or enable intuitive, non-complex interfaces to quickly execute complex user requests with improved access, privacy, and security enforcement. The enterprise generative artificial intelligence system 402 can include a human-machine interface for receiving natural language queries and presenting relevant information with predictive analytics from the enterprise information environment in response to these queries. For example, the enterprise generative artificial intelligence system 402 can understand the language, intent, and / or context of the user's natural language query. The enterprise generative artificial intelligence system 402 can execute the user's natural language query to discern relevant information from the enterprise information environment to present to the human-machine interface (e.g., in the form of an "answer").

[0047] In some embodiments, the generative AI models (e.g., the orchestrator's large language model) of enterprise generative AI system 402 can interact with agents (e.g., retrieval agents, retriever agents) to retrieve and process information from various data sources. For example, a data source can store data records and / or segments of data records that can be identified by enterprise generative AI system 402 based on embedded values ​​(e.g., vector values ​​associated with the data records and / or segments). Data records can include tables, text, images, audio, video, code, and / or application output (e.g., predictive analysis and / or other insights generated by an AI application), among other things.

[0048] In some embodiments, enterprise generative artificial intelligence system 402 can generate context-based synthetic output based on information retrieved from one or more retriever models. For example, a retriever model (e.g., a retriever model or a retrieval agent) can provide additional retrieved information to a large language model to generate additional context-based synthetic output until context validation criteria are met. Once the validation criteria are met, enterprise generative artificial intelligence system 402 can output the additional context-based synthetic output as a result or set of instructions (collectively, "answers").

[0049] In various embodiments, enterprise generative AI system 402 provides transformed context-based intelligent generation results. For example, enterprise generative AI system 402 can use a natural language interface to process input from enterprise users to quickly locate, retrieve, and present relevant data across the entire corpus of the enterprise's information systems.

[0050] As discussed elsewhere herein, the enterprise generative AI system 402 can process both machine-readable input (e.g., compiled code, structured data, and / or other types of formats that can be processed by a computer) and human-readable input. The input can also include complex input, such as input that includes "and," "or," and / or input that includes different types of information to satisfy the input (e.g., data records, text documents, database tables, and AI insights). In one example, a complex input might be "How many different engineers has John Doe worked with within his engineering department?" This may require the enterprise generative AI system 402 to identify John Doe in a first iteration, identify John Doe's department in a second iteration, determine the engineers in that department in a third iteration, then determine which of those engineers John Doe has interacted with in a fourth iteration, and then finally combine these results, or portions thereof, to generate a final answer to the query. More specifically, the enterprise generative AI system 402 can use portions of the results of each iteration to generate contextual information (or simply context), which can then inform subsequent iterations.

[0051] Enterprise systems 404 may include enterprise applications (e.g., artificial intelligence applications), enterprise data stores, client systems, and / or other systems of the enterprise information environment. As used herein, an enterprise information environment may include one or more networks (e.g., cloud, local, air gap, or other) of enterprise systems (e.g., enterprise applications, enterprise data stores), and client systems (e.g., computing systems for accessing enterprise systems). Enterprise systems 404 may include different computing systems, applications, and / or data stores, as well as enterprise-specific requirements and / or features. For example, enterprise systems 404 may include access and privacy controls. For example, an organization's private network may include an enterprise information environment containing various enterprise systems 404. Enterprise systems 404 may include, for example, CRM systems, EAM systems, ERP systems, FP&A systems, HRM systems, and SCADA systems. Enterprise systems 404 may include or utilize artificial intelligence applications, and artificial intelligence applications may utilize enterprise systems and data. Enterprise systems 404 may include data flows and management of different processes (e.g., one or more organizations), and may provide access to the enterprise's systems and users while preventing access from other systems and / or users. It will be understood that in some embodiments, references to an enterprise information environment may also include an enterprise system, and references to an enterprise system may also include an enterprise information environment. In various embodiments, the functionality of enterprise system 404 may be performed by one or more servers (e.g., cloud-based servers) and / or other computing devices.

[0052] External systems 406 may include applications, data stores, and systems external to the enterprise information environment. In one example, enterprise system 404 may be a portion of an organization's enterprise information environment that is inaccessible to users or systems external to the enterprise information environment and / or the organization. Thus, example external systems 406 may include Internet-based systems external to the enterprise information environment, such as news media systems and / or social media systems. In various embodiments, the functionality of external systems 406 may be performed by one or more servers (e.g., cloud-based servers) and / or other computing devices.

[0053] The communication network 408 may represent one or more computer networks (e.g., LANs, WANs, air-gapped networks, and / or cloud-based networks, etc.) or other transmission media. In some embodiments, the communication network 408 may provide communication between systems, modules, engines, generators, layers, agents, tools, orchestrators, data stores, and / or other components described herein. In some embodiments, the communication network 408 includes one or more computing devices, routers, cables, buses, and / or other network topologies (e.g., meshes, etc.). In some embodiments, the communication network 408 may be wired and / or wireless. In various embodiments, the communication network 408 may include a local area network (LAN), a wide area network (WAN), the Internet, and / or one or more networks that may be public, private, IP-based, non-IP-based, air-gapped, and / or the like.

[0054] Figure 5 A diagram depicts an example enterprise generative artificial intelligence system 402, according to some embodiments. Figure 5 In the example of FIG, the enterprise generative artificial intelligence system 402 includes a management module 502, an orchestrator module 504, a retrieval agent module 506-1, an unstructured data retriever agent module 506-2, a structured data retriever agent module 506-3, a type system retriever agent module 506-4, a machine learning insight module 506-5, a time series processing agent 506-6, an API agent module 506-7, a mathematical agent module 506-8, a visualization agent module 506-9, a code generation agent module 506-10, an unstructured data retrieval tool 508-1, a structured data retrieval tool 508-2, a text processing tool module 508-3, an image processing tool module 508-4, a time series processing tool module 508-5, an API tool module 508-6, and a visualization tool module 508-9. tool module 508-7, optimizer tool module 508-8, filter tool module 508-9, projection tool module 508-10, grouping tool module 508-11, sorting tool module 508-12, restriction tool module 508-13, code generation tool module 508-14, understanding module 510, chunking module 512, enterprise access control module 514, artificial intelligence traceability module 516, parallelization module 520, model generation module 522, model deployment module 524, model optimization module 526, interface module 528, communication module 530, (one or more) vector data store 540, (one or more) model registry data store 550, (one or more) feature data store 560 and (one or more) enterprise generative artificial intelligence system data store 570.

[0055] Management module 502 can function to (e.g., create, read, update, delete, or otherwise access) data associated with enterprise generative artificial intelligence system 402. Management module 502 can store or otherwise manage or store data in any of data stores 540-570 and / or one or more other local and / or remote data stores. It will be understood that the data store can be a single data store local to enterprise generative artificial intelligence system 402 and / or multiple data stores remote from enterprise generative artificial intelligence system 402. In some embodiments, the data stores described herein include one or more local and / or remote data stores. Management module 502 can operate manually (e.g., through user interaction with a GUI) and / or automatically (e.g., triggered by one or more of modules 504-530). As with other modules described herein, some or all of the functionality of management module 502 can be included in and / or collaborate with one or more other modules, systems, and / or data stores.

[0056] The orchestrator module 504 can function to generate and / or execute one or more orchestrator agents (or simply orchestrators). The orchestrator can orchestrate, supervise, and / or otherwise control agents 506. In some implementations, the orchestrator includes one or more large language models. The orchestrator can interpret inputs, select appropriate agents for handling queries and other inputs, and route the interpreted inputs to the selected agents. The orchestrator can also perform various supervisory functions. For example, the orchestrator can implement stop conditions to prevent the understanding module from falling into an infinite loop during the iterative context-based generative artificial intelligence process. The orchestrator can also include one or more other types of models to process (e.g., transform) non-text inputs. In addition to or in place of the large language models used for some or all of the agents and / or modules described herein, other models (e.g., other machine learning models, translation models) can also be included.

[0057] In some embodiments, the orchestrator can process data received from various data sources in different formats (which can be processed using natural language processing (NLP) (e.g., using tokenization, stemming, lemmatization, and normalization, etc.) with vectorized data) and can generate pre-trained transformers that are fine-tuned or retrained on specific data tailored to the associated data domain or data application (e.g., SaaS applications, legacy enterprise applications, artificial intelligence applications). Further processing can include data modeling feature inspection and / or machine learning model simulation to select one or more appropriate analysis channels. Example data objects can include accounts, products, employees, suppliers, opportunities, contracts, locations, digital portals, geo-located manufacturers, supervisory control and data acquisition (SCADA) information, open manufacturing system (OMS) information, inventory, supply chain, bill of materials, transportation services, maintenance logs, and service logs.

[0058] In some embodiments, the orchestrator module 504 can use various components, as needed, to inventory or generate objects (e.g., components, functionality, and / or data, etc.) using rich descriptive metadata to dynamically generate embeddings for developing knowledge across a wide range of data domains (e.g., documents, tabular data, insights derived from artificial intelligence applications, web content, or other data sources). In an example implementation, the orchestrator module 504 can utilize some or all of the components described herein, for example. Thus, for example, the orchestrator module 504 can facilitate storage, transformation, and communication to facilitate processing and embedding data. In some implementations, the orchestrator module can create embeddings for multiple data types across multiple vertical industries and knowledge domains, as well as specific enterprise knowledge. Knowledge can be explicitly modeled and / or learned by the orchestrator module 504, the agent 506, and / or the tool 508. In an example, the orchestrator module 504 (and / or the chunking module 512 discussed below) generates embeddings that are translated or transformed to be compatible with the understanding module 510.

[0059] In some embodiments, the orchestrator 504 can be configured to operate or interface different data domains with components of the enterprise generative artificial intelligence system 402. In one example, the orchestrator module 504 can embed objects from a specific data domain and across data domains, applications, data models, analytical byproducts, artificial intelligence predictions, and knowledge repositories to provide robust search functionality without requiring specialized programming for each different data domain or data source. For example, the orchestrator module 504 can create multiple embeddings for a single object (e.g., an object can be embedded in a domain-specific or application-specific context). In some embodiments, the chunking module 512 (discussed below) in conjunction with the orchestrator module 504 can orchestrate the data domain to embed objects of the data domain in the enterprise information system and / or environment. In some embodiments, the orchestrator 504 can collaborate with the chunking module 512 to provide the embedding functionality described herein.

[0060] In some embodiments, the orchestrator module 504 can cause the agent 506 to perform data modeling to translate the original source data format into a target embedding (e.g., object and / or type, etc.). The data format can include some or all of the different types or modalities described herein (e.g., multimodal, text, encoded, linguistic, statistical, audio, visual, audiovisual, etc.). In an example implementation, the orchestrator module 504 and / or the enterprise generative artificial intelligence system 402 typically employs a type system of a model-driven architecture to perform data modeling to translate the original source data format into the target type. The enterprise generative artificial intelligence system 402 and the knowledge base of the generative artificial intelligence model can create the ability to integrate or combine insights from different artificial intelligence applications.

[0061] As discussed elsewhere herein, in addition to human-readable input, enterprise generative AI system 402 can also process machine-readable input (e.g., compiled code, structured data, and / or other types of formats that can be processed by a computer). Input can also include complex input, such as input that includes "ands" and "ors," input that includes different types of information (e.g., text documents, database tables, and AI insights) to satisfy the input, etc. Orchestrator 504 can decompose these complex inputs (e.g., by using a large language model) for processing by multiple agents 506 (e.g., in parallel).

[0062] As discussed above, the orchestrator module 504 can function to perform and / or otherwise handle various supervisory functions. In some implementations, the orchestrator module 504 can enforce conditions (e.g., stopping conditions, resource allocation, and / or prioritization, etc.). For example, a stopping condition can indicate a maximum number of iterations (or hops) that can be performed before the iterative process terminates. The stopping conditions and / or other features managed by the orchestrator module 504 can be included in the large language model hints and / or the large language model of the orchestrator and / or the understanding module 510 discussed below. In some embodiments, the stopping condition can ensure that the enterprise generative artificial intelligence system 402 will not fall into an infinite loop. This feature can also allow the enterprise generative artificial intelligence system 402 to have the flexibility to have different numbers of iterations for different inputs (e.g., as opposed to having a fixed number of hops). In another example, the orchestrator module 504 can perform resource allocation based on computing conditions, such as virtualization or load balancing. In some implementations, orchestrator module 504 and / or agent 506 include a model that can convert (or transform) images, database tables, and / or other non-text input into a text format (eg, natural language).

[0063] In some embodiments, the orchestrator module 504 can function to collaborate with the agents 506 (e.g., the retrieval agent module 506-1, the unstructured data retriever agent module 506-2, and the structured data retriever agent module 506-3) to iteratively and non-iteratively process the input to determine an output result or answer, determine the context and root cause for informing subsequent iterations, and determine whether the large language model (e.g., of the orchestrator 504 and / or the understanding module 510) requires additional information to determine the answer. For example, the orchestrator module 504 can receive a query and instruct the agent 506-1 to retrieve the associated information. The retrieval agent module 506-1 can then select the unstructured data retriever agent module 506-2 and / or the structured data retriever agent module 506-3 based on whether the orchestrator module 504 wants to retrieve structured data records or unstructured data records. The appropriate agent 506 can select the corresponding tool and provide the tool output to the orchestrator module 504 and / or the understanding module 510 to determine the final result.

[0064] Orchestrator 504 can also select and swap models as needed. For example, in addition to before or after runtime, orchestrator 504 can also swap out models (e.g., data models, large language models, machine learning models) of enterprise generative artificial intelligence system 402 at runtime or during runtime. For example, orchestrator 504, agents 506, and understanding module 510 can use a particular set of machine learning models for one domain and other models for a different domain. Orchestrator 504 can select and use the appropriate model for a given domain and / or input.

[0065] In some embodiments, the orchestrator 504 can combine (e.g., concatenate) the outputs / results from the various agents to create a unified output. For example, one or more of the agent modules 506 can obtain / output a document (or (one or more) segments thereof) or related information (e.g., a text summary or translation), and another agent module 506 can obtain / output a database table, etc. The orchestrator 504 can then use one or more machine learning models (e.g., a large language model and / or another machine learning model) to combine the outputs / results into a unified output (e.g., having a common data format, such as natural language, etc.).

[0066] In some implementations, orchestrator 504 pre-processes input (e.g., initial input) before sending it to one or more agents 506 for processing. For example, orchestrator 504 may transform a first portion of the input into a SQL query and send it to the Unstructured Data Retriever Agent Module 506-2 agent, and transform a second portion of the input into an API call and send it to the API Agent Module 506-7, etc. In another example, such transformation functionality may be performed by agents 506 instead of or in addition to orchestrator 504.

[0067] The orchestrator module 504 can function to process, extract, and / or transform different types of data (e.g., text, database tables, images, videos, and / or code, etc.). For example, the orchestrator module 504 can accept a database table as input and transform it into natural language describing the database table, which can then be provided to the understanding module 510, which can then process the transformed input into an "answer" or otherwise satisfy the query. In some embodiments, a large language model can be used to process text, while another model can be used to convert (or transform) images, database tables, and / or other non-text input into a text format (e.g., natural language).

[0068] It will be understood that, in some embodiments, orchestrator module 504 may include some or all of the functionality of comprehension module 510. For example, comprehension module 510 may be a component of orchestrator module 504. Similarly, in some embodiments, comprehension module 510 may include some or all of the functionality of orchestrator module 504.

[0069] exist Figure 5In the example of , agent modules 506 include various different example agent modules 506-1 to 506-N. It will be understood that these are shown as examples and that various embodiments may include different agents instead of or in addition to agents 506-1 to 506-N. In some embodiments, each agent 506 includes hardware and / or software and includes one or more large language models, one or more other machine learning models and / or functions to provide reasoning functionality to complete a specified set of tasks. It will be understood that references to agent modules can refer to the agents themselves and / or components that generate and / or execute the agents. In some embodiments, an orchestrator is a type of agent and can be referred to as an orchestrator agent. Therefore, references to an orchestrator can refer to the orchestrator itself and / or components that generate and / or execute the orchestrator.

[0070] In some embodiments, the agent 506 uses the model to determine a selection sequence. The determined decision sequence may include comparing options, aggregating multiple options, and / or analyzing multiple options to generate contextual information related to the options. The agent 506 may also check for conflicts or similarities between options.

[0071] In various embodiments, some or all of the agents 506 can process data of different data types and / or data formats. For example, the agent module 506 can receive a database table or an image as input (e.g., received from the tool 508) and translate the table or image into natural language describing the table or image, which can then be output for processing by other modules, models, and / or systems (e.g., the composer module 504 and / or the understanding module 510). In one example, a large language model can be used to process text, while another model can be used to convert (or transform) images, database tables, and / or other non-text inputs into a text format (e.g., natural language).

[0072] The retrieval agent module 506-1 can function to retrieve structured and unstructured data records. In some embodiments, the retrieval agent module 506-1 can coordinate / instruct the unstructured data retriever agent module 506-2 to retrieve unstructured data records, and coordinate / instruct the structured data retriever agent module 506-3 and the type system retriever agent 506-4 to retrieve structured data records. For example, the retrieval agent module 506-1 can collaborate with other agents 506 and tools 508 to generate SQL queries to query an SQL database.

[0073] The unstructured data retriever agent module 506-2 can function to retrieve unstructured data records and / or paragraphs (or segments) of these data records (e.g., from an unstructured data store). Unstructured data records can include, for example, text data stored on a file system in formats such as PDF, DOCX, .MD, HTML, TXT, and PPTX.

[0074] In some embodiments, agent 506-2 can use embeddings (e.g., vectors stored in vector storage 540) when retrieving information. For example, agent 506-2 can use similarity evaluation or search on vector data storage 540 to find related data records based on k-nearest neighbors, where embeddings that are closer to each other are more likely to be related.

[0075] In some embodiments, the unstructured data retriever agent module 506-2 implements a read-extract-answer (REA) data retrieval process and / or a read-answer (RA) data retrieval process. More specifically, REA and RA may be appropriate when the system 402 needs to process large amounts of data. For example, a query may identify many different data records and / or paragraphs (e.g., hundreds or thousands of data records and paragraphs). For simplicity, references to data records may include data records and / or paragraphs.

[0076] More specifically, the unstructured data retriever agent module 506-2 can determine whether each data record is relevant to answering the query and filter out irrelevant data records. For example, the agent 506-2 can calculate and assign a relevance score for each retrieved data record (e.g., using a machine learning relevance model). The relevance score can be relative to other retrieved data records. For example, the least relevant data record can be assigned a minimum value (e.g., 0), and the most relevant data record can be assigned a maximum value (e.g., 100). The unstructured data retriever agent module 506-2 can filter out relevant documents (or irrelevant documents). For example, the unstructured data retriever agent module 506-2 can filter out data records with a relevance score below a configurable threshold (e.g., 50). In some embodiments, the number of data records that the unstructured data retriever agent module 506-2 can retrieve for a particular input or query can be user- or system-defined and can also be configurable. For example, the system can define that a maximum of 50 data records can be returned.

[0077] In some embodiments, a large language model (e.g., of the unstructured data retriever agent module 506-2) can identify key points of relevant documents and paragraphs, and then provide the key points to the large language model (e.g., the large language model of the arranger 504). The large language model can provide summaries that can be used to generate query answers (e.g., the summaries can be query answers). This can, for example, allow the system 402 to view a wide variety of concepts and documents (e.g., as opposed to an iterative process). In some embodiments, if the number of documents or paragraphs is below a threshold, the unstructured data retriever agent module 506-2 can skip the "extraction" step (e.g., summarizing key points) and provide the paragraphs directly to the large language model. This can be referred to as an RA process.

[0078] The structured data retriever agent module 506-3 can function to retrieve structured data records and / or their segments (or fragments) from various structured data stores. For example, a structured data record can include tabular data persisted in a relational database, a key-value store, or an external database and modeled or accessed using entity types (or simply types). A structured data record can include a data record structured according to one or more data models (e.g., a complex data model) and / or a data record that can be retrieved based on one or more data models. A structured data record can include a data record stored in a structured data store (e.g., a data store structured according to one or more data models).

[0079] In certain implementations, the data model may include a graph structure of objects or types, and agents 506 and / or tools 508 may traverse the graph in different paths to identify relevant types of the data model (e.g., depending on the query and the plan provided by orchestrator module 504 to answer the query), and may combine multiple tables using complex joins (e.g., as opposed to simply passing a single piece of data from the single table and operating on the single table). The paths may be stored in a data store (e.g., vector data store 540) for efficient retrieval.

[0080] In some embodiments, the structured data retriever agent module 506-3 can use a variety of different tools to retrieve structured data (e.g., structured data retrieval tool 508-2, filter tool 508-9, projection tool 508-10, grouping tool 508-11, sorting tool 508-12, and restriction tool 508-13, etc.). In some embodiments, once the structured data retriever agent module 506-3 has traversed the data model and retrieved (one or more) relevant types and / or subsets of the data model, the structured data retriever agent module 506-3 can then use this information and agent and / or tool output to construct a structured query specification that the structured data retriever agent module 506-3 can execute against one or more structured data stores to retrieve structured data records.

[0081] The type system retriever agent module 506-4 can function to retrieve types, data models, and / or subsets of data models. For example, a data model can include a variety of different types, and each type can describe a data field, an operation, and a function. Each type can represent a different object (e.g., a real-world object such as a machine or sensor in a factory, etc.), and each type can include a large language model context that provides context for a large language model. Types can be defined in a natural language format for efficient processing by a large language model.

[0082] In some embodiments, the type system is designed to be used by different computing systems, application developers, data scientists, operations personnel, and / or other users to build applications, develop and execute machine learning algorithms, and manage and monitor the status of jobs running on the type system (e.g., in some embodiments, the enterprise generative artificial intelligence system). A design goal of the type system is to provide a framework that enables systems, application developers, data scientists, and other users to communicate with each other efficiently using the same language. Thus, application developers can interact with the enterprise generative artificial intelligence system 402 in the same way as data scientists. For example, they can use the same types, the same methods, and the same features.

[0083] In some embodiments, the type system can abstract the complex infrastructure within enterprise generative AI system 402. In one example, developers may never need to write SQL, CQL, or some other query processing language to access data. When a user reads data, enterprise generative AI system 402 can generate the correct query for the underlying data store, submit the query to the database, and present the results back to the user in the form of an object or a collection of results.

[0084] In some embodiments, a type can be similar to a programming language class (e.g., a Java class) and describe data fields, operations, and functions (e.g., static functions) that can be called on the type or by one or more applications, but the type is not tied to any specific programming language. A type can be a definition of one or more complex objects that the system 402 can understand. For example, a type can represent a wide range of objects, such as a water pump. In addition to objects, types can also be used to model systems (e.g., computing clusters, key-value data stores, file systems, file stores, and enterprise data stores). In some embodiments, complex relationships such as "when which light bulbs are in which light fixtures" can be modeled as types.

[0085] A type definition can be a canonical type declared in metadata using a syntax similar to that used by types persisted in relational or NoSQL data stores. The canonical model in the type system is an application-independent model, enabling all applications to communicate with each other in a common format. Unlike standard types, a canonical type consists of two parts: a canonical type definition and one or more transformation types. The canonical type definition defines the interface for integration, and the transformation type is responsible for transforming the canonical type into the corresponding type. Using the transformation type, the integration layer can transform the canonical type into the appropriate type.

[0086] A type system provides data accessibility, compatibility, and operability with different systems and data. Specifically, a type system addresses data operability across various programming languages, inconsistent data structures, and incompatible software application programming interfaces. A type system provides a data abstraction that defines an extensible type model that enables the dynamic addition of new properties, relations, and functions without expensive development cycles. A type system can be used as a domain-specific language (DSL) within a platform that developers, applications, or UIs use to access data. A type system provides the ability to interact with data to process, predict, or parse based on one or more type or function definitions within the type system.

[0087] The machine learning insight agent module 506-5 can function to obtain and / or process output from an artificial intelligence application (e.g., artificial intelligence application insights). For example, the machine learning insight module 506-5 can instruct the text processing tool 508-3 to perform a text processing task (e.g., converting an artificial intelligence application into natural language), instruct the image processing tool 508-4 to perform an image processing task (e.g., generating a natural language summary of an image output from an artificial intelligence application), instruct the time series tool 508-3 to summarize time series data (e.g., time series data output from an artificial intelligence application), and instruct the API tool 508-6 to perform an API call task (e.g., executing an API call to trigger or access an artificial intelligence application).

[0088] The time series processing agent 506 - 6 may function to obtain and / or process time series data, such as time series data output from various applications (e.g., artificial intelligence applications), machines, sensors, etc. The time series processing agent 506 - 6 may instruct and / or collaborate with the time series processing tool module 508 - 3 to obtain time series data from one or more artificial intelligence applications and / or other data sources.

[0089] The API agent module 506-7 can function to coordinate and manage communications with other applications. For example, the API agent module 506-7 can instruct the API tool module 508-6 to execute various API calls and then process the tool outputs (e.g., transform into natural language summaries).

[0090] The mathematical agent module 506-8 can function to determine whether the agent 506 or the large language model requires additional information to generate an answer or result. In some embodiments, the mathematical agent 506-8 can instruct the optimizer tool module 508-8 to perform various prescriptive parsing functions and mathematical optimizations to assist in the calculation of answers to various questions. For example, the orchestrator module 504 can use the mathematical agent module 506-8 to generate a plan and determine whether the orchestrator module 504 requires more information to generate a final result.

[0091] The visualization agent module 506-9 can function to generate one or more visualizations and / or graphical user interfaces, such as dashboards and charts. For example, the visualization agent module 506-9 can execute the visualization tool module 508-7 to generate a dashboard based on information retrieved by other agents and / or information output by other agents (e.g., a natural language summary of an associated tool output). The visualization agent module 506-9 can also function to generate summaries of visual elements (e.g., natural language summaries), such as charts, tables, and images.

[0092] The code generation agent module 506-10 can function to instruct the code generation tool module 508-14 to generate source code, machine code, and / or other computer code. For example, the code generation agent module 506-10 can be configured to determine what code is needed (e.g., to satisfy a query and create an application, etc.) and instruct the tool 508-14 to generate the code in a specific language or format.

[0093] In some embodiments, tools 508 are specific functions that an agent (e.g., agent 506, orchestrator module 504) can access or execute when attempting to complete (one or more) specified tasks (e.g., in a set of specified tasks of a plan determined by orchestrator module 504). Tools 508 can include software and / or hardware. Tools 508 can also include one or more machine learning models, but they can also include functionality without any machine learning models. In some embodiments, tools 508 do not include large language models, but in other embodiments, the tools can include large language models. In some embodiments, some or all of agents 506 and / or tools 508 can be manually configured (e.g., by a user). Agents 506 and tools 508 can also normalize data (e.g., to a common data format) before outputting the data.

[0094] Unstructured data retrieval tool 508-1 can function to retrieve unstructured data records from an unstructured data store. In some embodiments, agent 506-2 can use embeddings (e.g., vectors stored in vector storage 540) when retrieving information. For example, agent 506-2 can use similarity evaluation or search to find related data records based on k-nearest neighbors, where embeddings that are closer to each other are more likely to be related. Structured data retrieval tool 508-2 can function to access and retrieve structured data records from a structured data store (e.g., structured or modeled according to a data model). Structured data retrieval tool 508-2 can be executed by structured data retriever agent module 506-3).

[0095] The text processing tool module 508-3 can function to retrieve and / or transform text (e.g., from unstructured data records) and perform other text processing tasks (e.g., transforming text-based output of an artificial intelligence application into natural language). The image processing tool module 508-4 can function to perform image processing tasks (e.g., generating a natural language summary of an image). The time series processing tool module 508-5 can function to obtain and / or process time series data (e.g., output from an artificial intelligence application, a sensor, etc.). For example, the time series processing tool module 508-3 can be executed by one or more of the agents 506 to obtain and process time series data. The API tool module 508-6 can function to perform API call tasks (e.g., executing an API call to trigger or access an artificial intelligence application). For example, a different agent 506 can use the API tool module 506-8 each time the agent needs to access or trigger another application.

[0096] The visualization tool module 508-7 can function to generate one or more visualizations and / or graphical user interfaces, such as dashboards, etc. For example, the visualization tool module 508-7 can generate dashboards based on information retrieved by other agents and / or information output by other agents (e.g., a natural language summary of associated tool output). The filter tool module 508-9 can function to filter data records and / or types, etc. For example, the filter tool module 508-9 can filter projections (e.g., fields) identified by the projection tool module 508-10 as part of the structured data retrieval process. In various embodiments, the tools 508 can be executed in parallel or otherwise.

[0097] In some embodiments, the filter tool module 508-9 can identify implicit filters based on a query or other input, and those identified implicit filters can be used as part of a structured data retrieval process. For example, a query may include "When was the last time that premium towels sold out?" The filter tool module 508-9 can identify "premium towels" as a filter (e.g., based on an associated type description). The filter tool module 508-9 can also identify contextual datetime filters. For example, a query may include "How many systems were offline yesterday?" The filter tool module 508-9 can determine yesterday's date while taking into account time zones and other relevant data to generate accurate filters. In some embodiments, the filter tool module 508-9 can validate the identified filters before using them (e.g., as part of a structured data retrieval process).

[0098] The projection tool module 508-10 can function to identify and select fields (e.g., type fields, object fields) relevant to determining an answer to a query or other input. The grouping tool module 508-11 can function to group data (e.g., type and tool output, etc.), which can then be used (e.g., by the structured data retriever agent module 506-3 and / or the structured data retrieval tool module 508-2) to generate a structured query request.

[0099] The sorting tool module 508-12 can function to sort data (e.g., by type and tool output, etc.), which can then be used (e.g., by the structured data retriever agent module 506-3 and / or the structured data retrieval tool module 458-2) to generate a structured query request. The limiting tool module 508-13 can function to limit the output of the structured data retrieval process. For example, it can limit the number of retrieved data records, types, groups, and / or filters, etc.

[0100] The code generation tool module 508-14 can function to generate source code, machine code, and / or other computer code. For example, the code generation tool module 508-14 can be configured to generate and / or execute SQL queries and / or JAVA code, etc. The code generation tool module 508-14 can be used to facilitate query generation for agents, other tools, and large language models, etc. In some embodiments, the code generation tool module 508-15 can be configured to generate source code for an application or create an application.

[0101] The understanding module 510 can function to process inputs to determine a result (e.g., an "answer"), determine the root cause of the result, and determine whether the understanding module 510 requires more information to determine the result. The understanding module 510 can output information (e.g., results or additional queries) in a natural language format or a machine language format. In some implementations, features of one or more models of the understanding module define conditions or functions for determining whether more information is needed to satisfy the initial input or whether sufficient information exists to satisfy the initial input.

[0102] In some embodiments, the understanding module 510 includes one or more large language models. The large language model can be configured to generate and process context and other information described herein. The understanding module 510 may also include other language models that pre-process the input (e.g., user query) before the input is provided to the agent for processing. The understanding module 510 may also include one or more large language models that process the outputs from other models and modules (e.g., the model of the agent 506). The understanding module 510 may also include another large language model that is used to process the answer from a large language model into a format that is more consistent with the final answer that can be transmitted to various users and / or systems (e.g., the user or system that provided the initial query or other intended recipient of the answer).

[0103] For example, the understanding module 510 can format answers based on various viewpoints. The viewpoints can be based on the type of user (e.g., human or machine), the user's role (e.g., data scientist, engineer, director, etc.), and access permissions. Thus, the viewpoints enable the understanding module 510 to generate and provide answers that are specifically tailored to the recipient. The understanding module 510 can also notify the user and the system if it cannot find an answer (e.g., as opposed to presenting an answer that is likely to be erroneous or biased).

[0104] In some implementations, features of one or more large language models of the understanding module 510 define conditions or functions for determining whether more information is needed to satisfy the initial input or whether sufficient information exists to satisfy the initial input. The large language model of the understanding module 510 may also define a stopping condition indicating a stopping threshold condition that indicates a maximum number of iterations that can be performed before the iterative process terminates.

[0105] In some embodiments, the understanding module 510 can generate and store root causes and context (e.g., in the data store 910). The root cause can be an inference used by the understanding module 510 to determine an output (e.g., a natural language output, an indication that it needs more information, an indication that it can satisfy the initial input). The understanding module 510 can generate the context based on the root cause. In some implementations, the context includes a concatenation and / or annotations of one or more segments of the data record, and / or embeddings associated therewith and a mapping of the concatenation and / or annotations. For example, the mapping can indicate relationships between different segments and / or weighted or relative values ​​associated with different segments, etc. The root cause and / or context can be included in the hints provided to the large language model.

[0106] In some embodiments, the understanding module 510 includes a query and root cause generator that generates queries or other inputs for a model (e.g., a large language model, other machine learning model) and / or generates and stores root causes and context (e.g., in a data store 560). The query and root cause generator can function to process, extract, and / or transform different types of data (e.g., text, database tables, images, videos, and / or code, etc.). For example, the query and root cause generator can accept a database table as input and transform it into natural language describing the database table, which can then be provided to one or more other models (e.g., a large language model) of the understanding module 510, which can then process the transformed input into an “answer” or otherwise satisfy the query. In some implementations, the query and root cause generator includes a model that can convert (or transform) images, database tables, and / or other non-textual input into a textual format (e.g., natural language). It will be understood that although queries are used in various examples herein, other types of input (e.g., instruction sets) can be processed in the same or similar manner as described for queries.

[0107] In some embodiments, the understanding module 510 can use different models for different domains. For example, different domains can correspond to different industries (e.g., aerospace, defense), different technical environments (e.g., air-gapped, cloud-native), and / or different enterprises or organizations, etc. Thus, the understanding module 510 can use a specific model (e.g., a data model and / or a large language model) for a specific domain (e.g., a data model that describes the properties and relationships of aerospace objects and a large language model trained on an aerospace-specific dataset), and another data model and / or large language model for another domain (e.g., a data model that describes the properties and relationships of defense-specific objects and a large language model trained on a defense-specific dataset), and so on.

[0108] In some embodiments, orchestrator module 504 includes some or all of the functionality and / or structure of comprehension module 510 and / or 906, described further below. Similarly, in some embodiments, comprehension module 510 may include some or all of the functionality and / or structure of orchestrator module 504.

[0109] In some embodiments, the understanding module 510 can function to generate large language model prompts (or simply prompts) and prompt templates. For example, the understanding module 510 can generate a prompt template for processing initial input, a prompt template for processing iterative input (i.e., input received during the iterative process after processing the initial input), and another prompt template for the output result stage (i.e., when the understanding module 510 has determined that it has enough information and / or meets the stopping condition). The understanding module 510 can modify the appropriate prompt template depending on the stage of the iterative process. For example, the prompt template can be modified to generate prompts that include root causes and context, which can inform subsequent iterations.

[0110] Chunking module 512 can function to process (e.g., chunk) a corpus of data records (e.g., from one or more enterprise systems) for processing by enterprise generative artificial intelligence system 402. As used herein, data records can include any type of data record that can be stored in a data store, such as unstructured data records and structured data records. For example, data records can include documents (e.g., PDF, text, HTML, Markdown source code), database tables, information generated by an application (e.g., artificial intelligence application insights), images, audiovisual files, executable files, and / or data records structured according to a data model and / or type system. More specifically, chunking module 512 can pre-process and chunk data records. The chunking process can partition the data records and insert or append corresponding headers for each chunk. The header can, for example, include one or more attributes describing the chunk (e.g., the type of data record, the size of the chunk, etc.). Chunking can be referred to herein as segments. A segment can, for example, include a header as well as a paragraph of a text document, a portion of a database table, etc. For simplicity, a reference to a paragraph may include a segment and / or the content of the segment (e.g., text). The segment may be stored in a segment data store (e.g., vector store 540). The data record may be chunked into a tree structure where each leaf corresponds to a segment. The chunking may be rule-based.

[0111] In some implementations, preprocessing includes generating contextual information for data records and / or segments. Contextual information can improve security and the accuracy and reliability of associated retrieval operations. In one example, the contextual information includes contextual metadata. The contextual information can include references between segments and / or data records. For example, references can indicate relationships that can be used (e.g., traversed) when performing similarity assessments or other aspects of retrieval operations (e.g., by one or more of the agents 506). The contextual information can also include information that can assist large language models in generating plans and / or answers. For example, the chunking module 512 can generate contextual information for a structured data chunk (or paragraph) that includes, among other things, a natural language description of a data record and the location of related data records.

[0112] Context information can include access control. In some implementations, context information provides user-based access control. More specifically, context information can indicate which user roles can access the corresponding segment and / or data record, and / or which user roles cannot access the corresponding segment and / or data record. Context information can be stored in the header of the data record and / or data record segment.

[0113] In some embodiments, the chunking module 512 can generate embeddings based on both structured and unstructured data records and / or segments. The chunking module 512 can include a deep learning model that can convert and / or transform the data records into vector representations in which vectors of semantically similar data records (e.g., the content of the data records) are close together in the vector space. This can facilitate retrieval operations using the agent 506 and the tool 508.

[0114] In some embodiments, the chunking module 512 can generate embeddings using one or more embedding models (e.g., an implementation of the ColBERT embedding model). Embeddings can include digital representations of unstructured and / or structured data records that capture the semantic or contextual meaning of the data records. For example, an embedding can be represented by one or more vectors. The embeddings can be used when retrieving data records and performing similarity assessments or other aspects of the retrieval operation. The embeddings can be stored in an embedding index (e.g., a vector data store 540). In some embodiments, the vector store 540 is a type of database that is specifically optimized for storing embeddings and retrieving embeddings using a similarity heuristic (e.g., an approximate nearest neighbor (ANN) algorithm) that can be implemented by the agent 506 and / or the tool 508. In one example, the vector store 540 can include an implementation of the FAISS vector store.

[0115] In some implementations, the chunking module 512 generates an enhanced embedding. For example, the chunking module 512 can generate the enhanced embedding based on contextual information, data records, and / or data record segments. The enhanced embedding can include a vector value based on the embedding vector and the contextual information. In some embodiments, the enhanced embedding includes the embedding vector value and contextual metadata including the contextual information. The enhanced embedding can be indexed in an enhanced embedding data store (e.g., vector data store 540). The agent 506 and / or the tool 508 can retrieve unstructured and / or structured data records based on the enhanced embedding.

[0116] In some embodiments, the chunking module 512 can perform some or all of the functionality described herein periodically (e.g., in batches), on-demand, and / or in real-time. For example, the chunking module 512 can trigger the chunking described herein periodically, on-demand, manually, and / or automatically. In some implementations, subsequent chunking operations can incorporate only changes (e.g., "deltas") relative to previous chunking operations.

[0117] In some embodiments, the chunking module 512 can generate contextual information for data records and / or segments. The contextual information can be represented by contextual metadata that provides access control (e.g., role-based access control) to associated data records and / or segments. The contextual information can maintain references between data records and / or data record segments. The chunking module 512 can insert and / or append contextual information to the segment header. The chunking module 512 can generate contextual information before, after, or simultaneously with generating the associated embedding. For example, the contextual information can be used to create the embedding, or the contextual information can be used to enhance the embedding. The contextual information can be used by the chunking module 512 to map relationships between data records and / or segments of one or more enterprises or enterprise systems and store these relationships in a data model and / or data store (e.g., data store 570). As discussed elsewhere herein, the chunking module 512 can generate embeddings and / or enhance embeddings. In one example, the chunking module 512 implements the word2vec algorithm. In some implementations, the chunking module 512 utilizes a model trained on a domain-specific (or industry-specific) dataset.

[0118] Enterprise access control module 514 can function to provide enterprise access control (e.g., layers and / or protocols) for enterprise generative artificial intelligence system 402, associated systems (e.g., enterprise systems), and / or environments (e.g., enterprise information environments). Enterprise access control module 514 can provide functionality for enforcing access control policies for generating results (e.g., preventing orchestrator module 504 and / or comprehension module 510 from generating results that include sensitive information) and / or filtering generated results before providing final results.

[0119] In some implementations, the enterprise access control module 514 can evaluate whether a user is authorized to access all or only a portion of the results (e.g., answers). For example, a user can provide a query associated with a first department or subunit of an organization. Members of that department or subunit can be restricted from accessing certain data, data types, data models, or other aspects of the data domain to be searched. In cases where the initial results include data that the user has restricted access to, the enterprise access control module 514 can determine how to handle such restricted data, such as completely omitting the restricted data, omitting the restricted data but indicating that the results include data that the user has restricted access to, or providing information related to all of the initial results. In an example of completely omitting the restricted data, a final result set can be returned for presentation to the user, where the final result set does not inform the user that a portion of the initial results has been omitted. In an example of omitting the restricted data but providing the user with an indication that there is restricted data, the final results can include only those results that the user is authorized to access, but can include information indicating that there were X initial results but only Y results were output, where Y < X. In the third example above, all results, including those that the user has restricted access to, can be output to the user.

[0120] Additionally or alternatively, the enterprise access control module 514 can communicate with one or more other modules to obtain information that can be used to enforce access permissions / restrictions in conjunction with performing the retrieval operation rather than for controlling the presentation of results to the user. For example, the enterprise access control module 514 can restrict the data sources to which the retrieval operation is applied, such as not applying the retrieval operation to portions of the data source that are denied access to the user and applying the retrieval operation to portions of the data source that are permitted access to the user. Note that the above exemplary techniques for enforcing access restrictions have been provided for illustrative purposes and not by way of limitation, and it should be understood that modules operating in accordance with embodiments of the present disclosure can implement other techniques to present results via an interface based on access restrictions.

[0121] In some embodiments, to facilitate enforcing access restrictions in conjunction with searches conducted by enterprise generative artificial intelligence system 402, enterprise access control module 514 may store information associated with access restrictions or permissions for each user. To retrieve relevant restriction data for a user, enterprise access control module 514 may receive information identifying the user in conjunction with an input or when the user logs into the system on which enterprise access control module 514 is executing. Enterprise access control module 514 may use the information identifying the user to retrieve appropriate restriction data to support enforcing access restrictions in conjunction with enterprise searches. In some embodiments, enterprise access control module 514 may include credential management functionality of a model-driven architecture deployed with enterprise generative artificial intelligence system 402, or may be a remote credential management system communicatively coupled to enterprise generative artificial intelligence system 402 via a network.

[0122] The AI ​​traceability module 516 can function to provide traceability and / or explainability of answers generated by the enterprise generative AI system 402. For example, the AI ​​traceability module 516 can indicate the portion of the data record used to generate the answer and its associated data source. The AI ​​traceability module 516 can also function to validate the large language model output. For example, the AI ​​traceability module 516 can automatically and / or on-demand provide source references to validate or verify the large language model output. The AI ​​traceability module 516 can also determine the compatibility of different sources (e.g., data records, passages) used to generate the large language model output. For example, the AI ​​traceability module 516 can identify data records that contradict each other (e.g., one data record indicates that John Doe is an employee of Acme Company, while another data record indicates that John Doe works for a different company) and provide a notification that the output is a contradiction generated based on the conflicting information.

[0123] The parallelization module 520 can function to control the parallelization of the various systems, modules, agents, models, and processes described herein. For example, the parallelization module 520 can cause the parallel execution of different agents and / or orchestrators. The parallelization module 520 can be controlled by the orchestrator module 504.

[0124] The model generation module 522 can function to obtain, generate and / or modify some or all of the different types of models described herein (e.g., machine learning models, large language models, data models). In some implementations, the model generation module 522 can use various machine learning techniques or algorithms to generate models. As used herein, artificial intelligence and / or machine learning can include Bayesian algorithms and / or models, deep learning algorithms and / or models (e.g., artificial neural networks, convolutional neural networks), gap analysis algorithms and / or models, supervised learning techniques and / or models, unsupervised learning algorithms and / or models, semi-supervised learning techniques and / or models, random forest algorithms and / or models, similarity learning and / or distance algorithms, generative artificial intelligence algorithms and models, clustering algorithms and / or models, transformer-based algorithms and / or models, neural network transformer-based machine learning algorithms and / or models, and / or reinforcement learning algorithms and / or models, etc. Algorithms can be used to generate corresponding models. For example, an algorithm can be executed on a data set (e.g., a domain-specific data set, an enterprise data set) to generate and / or output a corresponding model.

[0125] In some embodiments, a large language model is a deep learning model (e.g., generated by a deep learning algorithm) that can recognize, summarize, translate, predict, and / or generate text and other content based on knowledge obtained from a large-scale data set. Large language models may include transformer-based models. Large language models may include Google's BERT, OpenAI's GPT-3, and Microsoft's Transformer. Large language models can process large amounts of data, thereby improving the accuracy of prediction and classification tasks. Large language models can use this information to learn patterns and relationships, which can help them make improved predictions and groupings relative to other machine learning models. Large language models may include artificial neural network transformers pre-trained using supervised and / or semi-supervised learning techniques. In some embodiments, large language models include deep learning models specifically for text generation. In some embodiments, large language models may be characterized by a large number of parameters (e.g., tens of billions or hundreds of billions of parameters) and a large text corpus for training them.

[0126] Although the systems and processes described herein use large language models, it will be understood that other embodiments may use different types of machine learning models instead of or in addition to large language models. For example, orchestrator 504 may use a deep learning model specifically designed to receive non-natural language input (e.g., images, videos, audio) and provide natural language output (e.g., summaries) and / or other types of output (e.g., video summaries).

[0127] Model deployment module 524 can function to deploy some or all of the different types of models described herein. In some implementations, model deployment module 524 can deploy models before or after deployment of an enterprise generative artificial intelligence system. For example, model deployment module 524 can collaborate with model optimization module 526 to exchange or otherwise modify a large language model of an enterprise generative artificial intelligence system.

[0128] In some implementations, the model registry 550 can store various models (e.g., machine learning models, large language models, data models) and / or model configurations. Models can be trained on general datasets and / or domain-specific datasets. For example, the model registry can store different configurations of various large language models (e.g., which can be deployed or exchanged within the enterprise generative artificial intelligence system 402). In some embodiments, each model can be associated with an embedded value or an enhanced embedded value to facilitate retrieval operations (e.g., in the same or similar manner as data record retrieval).

[0129] Model optimization module 526 can function to enable tuning and learning using the modules (e.g., understanding module 212) and / or models (e.g., machine learning models, large language models) described herein. For example, model optimization module 526 can tune understanding module 510 and / or orchestrator module 504 (and / or their models) based on tracking user interactions within the system, capturing explicit and / or implicit feedback (e.g., through a training user interface), and / or the like. In some example implementations, model optimization module 526 can use reinforcement learning to accelerate knowledge base bootstrapping. Reinforcement learning can be used to explicitly bootstrap various systems (e.g., enterprise generative artificial intelligence system 402) by instrumenting, for example, time spent and / or results clicked. Example aspects of model optimization module 526 include an innovative learning framework that can bootstrap models for different enterprise environments. Example aspects of model optimization module 526 can include an innovative learning framework that can bootstrap models for different enterprise environments.

[0130] In some embodiments, reinforcement learning is a machine learning training method based on rewarding desired behavior and / or punishing undesirable behavior. Typically, a reinforcement learning agent is able to perceive and interpret its environment, take actions, and learn through trial and error. Reinforcement learning uses algorithms and models to determine the optimal behavior in the environment to obtain the maximum reward. This optimal behavior is learned by interacting with the environment and observing how it responds. In the absence of a supervisor, the learner must independently discover the sequence of actions that maximizes the reward. This discovery process is similar to trial and error search. The quality of the actions is measured not only by the immediate rewards they return, but also by the delayed rewards they may obtain. Because it can learn actions that lead to ultimate success in an unseen environment without the help of a supervisor, reinforcement learning is a very powerful algorithm. ColBERT is an example retriever model that enables scalable BERT-based search on large text collections (e.g., within tens of milliseconds). ColBERT uses a late interaction architecture that uses BERT to independently encode queries and documents, and then adopts a "cheap" but powerful interaction step to model their fine-grained similarity. In addition to reducing the cost of re-ranking documents retrieved by traditional models, ColBERT’s pruning-friendly interaction mechanism also enables end-to-end retrieval directly from large document collections by leveraging vector similarity indices.

[0131] In some embodiments, the model optimization module 526 can retrain the model (e.g., the transformer-based natural language machine learning model) periodically, on demand, and / or in real time. In some example implementations, corresponding candidate models (e.g., candidate transformer-based natural language machine learning models) can be trained based on user selections, and the model optimization module 526 can replace some or all of the models with one or more candidate models that have been trained on the received user selections.

[0132] Note that the functionality described has been provided by way of non-limiting example, and other techniques may be used to generate queries and commands. For example, in additional or alternative implementations, a multimodal or generative pre-trained transformer that is an autoregressive language model using deep learning to produce human-like text can be used to generate queries from a search input (i.e., without using a seed library). For illustration, assume the search input is "What is the riskiest motor actuator at Tinker?" As with the above-described techniques, the search input can be subjected to entity extraction and mapping, which will result in an entity matching search input of "What is the riskiest WUC 11AAF at location AFB0123?" The input is subjected to embedding and vectorization, where a large language model is used for query generation, and the entity matching search input can be provided to a generative multimodal or large language model algorithm to generate a query. In such an implementation, contextual information, such as a schema defining metadata for table headers, field descriptions, and join keys, can be provided to the generative multimodal algorithm, which can be used to retrieve search results. For example, the schema can be used to translate the entity matching search input into a query (e.g., an SQL query).

[0133] The system can leverage the characteristics of a model-driven architecture (which uses rich, descriptive metadata to represent system objects (e.g., components, functionality, data, etc.) to dynamically generate queries for searching across a wide range of data domains (e.g., documents, tabular data, insights derived from AI applications, web content, or other data sources). Furthermore, the query module of an embodiment may be particularly well-suited for searches involving type systems whose objects are exposed through metadata. Thus, the query module disclosed herein represents an improvement in the field of query modules and provides a mechanism for rapidly deploying search functionality across various data domains relative to existing query modules, which are typically limited in their searchable data domains (e.g., a web query module is limited to web content, a file system query module is limited to searches of a file system, etc.).

[0134] In some embodiments, in addition to before or after runtime, the model optimization module 526 can also replace the model of the enterprise generative artificial intelligence system at runtime or during runtime. For example, the orchestrator module 504, the understanding module 510 and / or the agent 506 can use a specific set of machine learning models for one domain and use other models for a different domain. The model exchange module can select and use the appropriate model for a given domain. This can even occur during an iterative process. For example, when a new query is generated by the understanding module 510, the domain may change, which may trigger the model exchange module to select and deploy a different model appropriate for that domain.

[0135] In some embodiments, the model optimization module 526 can train the generative artificial intelligence model to develop different types of responses (e.g., best results, ranked results, smart cards, chatbots, and / or new content generation, etc.).

[0136] In some embodiments, the model optimization module 526 can retrain models (e.g., large language models) periodically, on demand, and / or in real time. In some example implementations, corresponding candidate models can be trained based on user selections, and the system can replace some or all of the models with one or more candidate models that have been trained on received user selections.

[0137] The interface module 528 can function to receive input (e.g., complex input) from a user and / or system. The interface module 528 can also generate and / or transmit output. The input can include system input and user input. For example, the input can include an instruction set, a query, a natural language input or other human-readable input, and / or a machine-readable input, etc. Similarly, the output can also include system output and human-readable output. In some embodiments, the input (e.g., a request, a query) can be input in various natural forms for human interaction (e.g., a basic text box interface, image processing, and / or voice activation, etc.) and processed to quickly find relevant and responsive information.

[0138] In some embodiments, interface module 528 may function to generate a graphical user interface component (e.g., a server-side graphical user interface component) that can be rendered as a complete graphical user interface on enterprise generative artificial intelligence system 402 and / or other systems. For example, interface module 528 may function to present an interactive graphical user interface for displaying and receiving information.

[0139] The communication module 530 may function to send requests, transmit and receive communications, and / or otherwise provide communications with one or more of the systems, modules, engines, layers, devices, data stores, and / or other components described herein. In certain implementations, the communication module 530 may function to encrypt and decrypt communications. The communication module 530 may function to send requests to and receive data from one or more systems via a network or portion of a network (e.g., the communication network 408). In certain implementations, the communication module 530 may send requests and receive data via a connection, all or part of which may be a wireless connection. The communication module 530 may request and receive messages and / or other communications from associated systems, modules, and / or layers, etc. The communications may be stored in the enterprise generative artificial intelligence system data store 570.

[0140] In some embodiments, the configuration, coordination, and collaboration of orchestrator module 504, agents 506, tools 508, and / or other modules of enterprise generative AI system 402 (e.g., understanding module 510) enable enterprise generative AI system 402 to provide a multi-hop architecture that enables complex reasoning across multiple agents 506, tools 508, and data sources (e.g., vector data stores, feature data stores, data models, enterprise data stores, unstructured data sources, structured data sources, etc.). In various embodiments, some or all of the modules of enterprise generative AI system 402 can be configured manually (e.g., by a user) and / or automatically (e.g., without requiring user input). For example, large language model hints can be configured, tool 508 descriptions can be configured (e.g., for more efficient utilization by agents 506 and orchestrator module 504), and a maximum number of hops or iterations can be configured. In one example, orchestrator module 504 receives a query from a user, and orchestrator module 504 determines a plan for answering the query and selects agents 506 and / or tools 508 to perform a set of prescribed tasks that formulate the plan to answer the query. Agents 506 and / or tools 508 perform the set of prescribed tasks, and orchestrator module 504 observes the results. Orchestrator module 504 determines whether to submit a final answer or whether orchestrator module 504 requires more information. If orchestrator module 504 has sufficient information, it can generate and / or provide a final answer. Otherwise, orchestrator module 504 can create another set of prescribed tasks, and the process can continue until orchestrator module 504 has sufficient information to answer or a stopping condition (e.g., a maximum number of hops) is met.

[0141] Figure 6 A flowchart 600 is depicted of an example generative artificial intelligence unstructured data and structured data retrieval process. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps may be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been included but could have been removed for clarity.

[0142] In step 602, a user provides a query that is received by orchestrator 604. Although a user query is shown here, it will be understood that other inputs (e.g., machine-readable input received from other systems) may be used instead. The orchestrator module may pre-process the user query. For example, orchestrator 604 may translate the machine-readable input into a natural language format, translate French language input into English language input, and so on.

[0143] In step 606, the orchestrator generates a query 606 based on the user query and selects various tools to process the orchestrator query 606. In step 608, the orchestrator 604 selects an unstructured data retrieval tool 610 and a structured query generator tool 612. The unstructured query tool 610 may query a data store 616 (e.g., a vector store) for relevant unstructured data records (step 614). The unstructured data records may be provided to another model 618 (e.g., a large language model of the orchestrator 604), which may generate a summary 620 of the retrieved unstructured data records. For example, the large language model of the orchestrator 604 may not be able to handle large data inputs, so the model 618 may generate a summary in a natural language format that can be efficiently and accurately processed by the orchestrator 604.

[0144] In step 622, structured query builder tool 612 queries type system 624 for relevant data. Query response 626 may identify one or more database tables and / or API calls from step 626, retrieve the database tables and / or types, and execute the API calls (e.g., for artificial intelligence applications). Visualization tool 628 may provide a visual summary 630 of the retrieved database tables and API call execution results. For example, visual summary 630 may include graphical elements (e.g., graphs, charts, dashboards) and / or natural language summaries of such graphical elements. In step 632, the orchestrator may generate a final result based on summary 620 (i.e., summaries based on unstructured data) and summary 630 (i.e., summaries based on structured data retrieval). The final result may include a natural language summary based on the user's perspective. The visualization may be three-dimensional (3D) and include, for example, interactive elements associated with deterministic outputs that enable execution of instructions (e.g., transmissions, control system commands, etc.), drilling down into traceability, activating application features, and the like.

[0145] In some embodiments, Figure 6The orchestrators, tools, and / or data stores described in

[0065] include some or all of the functionality of the orchestrators, agents, tools, and / or data stores described elsewhere herein. Thus, for example, orchestrator 604 may include some or all of the functionality of orchestrator 504, unstructured query tool 610 may include some or all of the functionality of unstructured data retriever agent module 506-2 and / or unstructured data retriever tool module 506-1, and so on.

[0146] Figure 7 A diagram 700 depicts an example logic flow for an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowchart illustrates a sequence of steps by way of example. It should be understood that some or all of the steps may be repeated, reorganized for parallel execution, and / or reordered, where applicable. Furthermore, for clarity, some steps may have been removed to avoid providing excessive information, and some steps may have been included but could have been removed for clarity.

[0147] In step 702, an orchestrator module (e.g., orchestrator module 504) receives an input (e.g., a complex input) and generates a plan and a corresponding set of prescribed tasks. The orchestrator generates several subqueries from the input, the plan, and / or the set of prescribed tasks. The orchestrator coordinates an unstructured data agent (e.g., an unstructured data retriever agent module 506-2) to handle a first subquery that may be based on a first portion of the set of prescribed tasks, coordinates a structured data agent (e.g., a structured data retriever agent module 506-3) to handle a second subquery that may be based on a second portion of the set of prescribed tasks, coordinates a time series processing agent module (e.g., a time series processing agent module 506-6) to handle a third subquery that may be based on a third portion of the set of prescribed tasks, and coordinates a visualization tool (e.g., a visualization agent module 506-9 and / or a visualization tool module 507-6) to generate a graphical representation of the final answer generated by the orchestrator. In some embodiments, the orchestrator may directly coordinate Figure 7 Other tools shown and / or indirectly coordinate other tools. For example, the orchestrator can directly coordinate (e.g., instruct) the agents, which can then directly coordinate their respective tools.

[0148] Steps 704-712 depict the generative artificial intelligence unstructured data retrieval and answering process (e.g., to answer the first sub-query). More specifically, in step 704, the unstructured data agent selects an unstructured data retrieval tool (e.g., unstructured data retrieval tool 508-1) to retrieve unstructured data records. In step 706, the unstructured data retrieval tool retrieves the unstructured data records from the vector data store 706 (e.g., using a similarity heuristic search, such as an ANN algorithm, etc.). In step 710, a large language model (e.g., a large language model of the unstructured data agent) extracts relevant paragraphs from the retrieved data records. In step 712, the large language model (e.g., a large language model of the unstructured data agent and / or the orchestrator) generates an answer to the first sub-query.

[0149] Steps 714-738 depict the generative artificial intelligence structured data retrieval and answering process (e.g., to answer the second sub-query). More specifically, in step 714, the structured data agent selects a type system retriever agent (e.g., type system retriever agent module 506-4). In step 716, in some embodiments, the type system retriever agent searches for types of data models in one or more data models in vector storage 718, and in step 720, the type system retriever agent (e.g., the type system agent's large language model) selects types of data models that are relevant to answering the initial query and / or the second sub-query. The retrieved types may include subsets (or sub-models) or data models that are relevant to the initial query and / or the second sub-query.

[0150] In step 722, a projection tool (e.g., projection tool module 508-10) selects a projection (e.g., a specific field of a relevant type) that is relevant to answering the initial query and / or the second subquery. The projection can be selected based on a type field document describing the type field from the data store 724. Based on the selected projection, the structured data agent can select different tools to generate a structured data retrieval specification query. Figure 7In the example of , the structured data agent selects a filter tool (e.g., filter 508-9) to filter the selected types based on the selected projection (step 726), selects a grouping tool (e.g., grouping tool module 508-11) to group types (e.g., filtered types) and / or projections, selects a restriction tool (e.g., restriction tool module 508-13) to restrict the structured data results, and selects a sorting tool (e.g., sorting tool 508-12) to sort the types and / or fields so that the structured data results are provided in a specific order. See steps 728, 730, and 734. In step 736, the structured data agent creates a structured data retrieval specification query based on the tool output of steps 726-734 and executes the query against the structured data store. In step 738, a large language model (e.g., a large language model of the structured data agent and / or the orchestrator) generates an answer (e.g., to the second subquery) based on the results of the query.

[0151] Steps 740-760 depict the generative artificial intelligence time series data retrieval and answering process. The generative artificial intelligence time series data retrieval and answering process can function in the same or similar manner as the generative artificial intelligence structured data retrieval and answering process, except that time series data is used instead of structured data. For example, a time series agent (e.g., time series agent 506-6) can be used instead of a structured data agent, and a time series processing tool (e.g., time series processing tool 508-5) can be used with a filter tool to construct a canonical query that can be used to retrieve time series data records (e.g., artificial intelligence application output) to generate an answer (e.g., an answer to the third subquery).

[0152] In step 762, the orchestrator may use a large language model that receives answers 712, 738, and 760 as input and generates results based on these inputs. A visualization tool may generate a graphical representation of the results (e.g., a graph, chart, dashboard, etc.).

[0153] Figure 8A A flowchart 800 is depicted illustrating an example iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowchart illustrates a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps may be repeated, reorganized for parallel execution, and / or reordered. Furthermore, for clarity, some steps may have been removed to avoid providing excessive information, and some steps may have been included but could have been removed for clarity.

[0154] In step 802, a user query is provided to a retrieval model (e.g., a retriever module of a retrieval agent module). In step 804, the retrieval model receives the query and performs a similar search (e.g., an ANN-based search) of vector storage 806. In step 810, the retrieved information is returned to the retriever model and provided to the large language model. In step 812, the large language model (e.g., the large language model used in step 810 and / or a different large language model) determines whether additional information is needed to answer the user query. If more information is needed, steps 804-812 may be iteratively repeated using updated large language model prompts and / or queries (e.g., using the large language model in step 808) until the large language model has sufficient information to answer the query or a stopping condition is met (e.g., a maximum number of iterations has been performed). In step 814, an answer is given (e.g., a final result if there is sufficient information for the large language model to determine an answer, or "I don't know" if the stopping condition is met before sufficient information can be received). The final result may also include the underlying reason used by the large language model to generate the answer. The large language models used in steps 808 and 810 may be the same large language model and / or different large language models.

[0155] Figure 8B A flowchart 830 is depicted of an example non-iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In these flowcharts and / or sequence diagrams, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate sequences of steps by way of example. It should be understood that, where applicable, some or all of the steps can be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been removed but included for clarity of illustration.

[0156] In step 832, a query is received. Query 832 is executed against vector store 834, and relevant paragraphs 836 are retrieved. In some embodiments, user query 832 is pre-processed (e.g., by an orchestrator) before being applied to vector store 834 to retrieve paragraphs 836. For example, query 832 may be translated, transformed, etc. Because vector stores may have difficulty handling complex input, the system may generate a new query or multiple shorter queries from user query 832 that vector store 834 can handle efficiently and accurately. Query 832 and paragraphs 836 are provided to a large language model 838, which may create extractions 840 (e.g., summaries of the paragraphs) for each paragraph. The extractions are combined (e.g., concatenated) in step 842 and provided to a large language model 844 along with query 832. In some embodiments, the extraction step is optional, and instead of extracting, the paragraphs may be concatenated and provided to a large language model 844. The large language model 844 may generate a final response based on query 832 and the combined extractions 842. In some embodiments, the large language model 844 can post-process the results before they are provided to the user (e.g., using an orchestrator). For example, it can be translated, formatted based on the viewpoint, include references and attributes, etc.

[0157] Figure 8C A flowchart 860 is depicted of an example non-iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In these flowcharts and / or sequence diagrams, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate sequences of steps by way of example. It should be understood that, where applicable, some or all of the steps can be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been removed but included for clarity of illustration.

[0158] In step 862, a query is received. For example, the query may be "How much wine do they produce?" This query is difficult for a traditional large language model to handle and will typically cause the large language model to hallucinate because it is unclear how to handle the "they" in the query 862. To address this issue, the enterprise generative artificial intelligence system can use context 864 to generate improved queries. For example, a previous conversation 864 (e.g., as part of a chat with a chatbot) may have included a conversation about France. The system can provide France as contextual information 864 to generate a new query 866, such as "How much wine does France produce?", which can prevent the large language model from hallucinating and allow the large language model to provide an accurate and reliable final result 878.

[0159] More specifically, the enterprise generative artificial intelligence system can generate a rewritten query 866 (e.g., using a large language model 865), which can be executed against a vector store 868 to retrieve a paragraph 870. In some embodiments, the rewritten query 866 can be preprocessed (e.g., by an orchestrator) before being applied to the vector store 868. For example, the rewritten query 866 can be translated, transformed, and the like. Because the vector store has difficulty handling complex input, the system can generate a new query or multiple shorter queries from the rewritten query 866 that the vector store 868 can handle efficiently and accurately. In some embodiments, this preprocessing can be performed when the rewritten query is generated (e.g., rewriting the query includes a preprocessing step).

[0160] The rewritten query 866 and the paragraphs 870 are provided to a large language model 869, which can create extractions 872 (e.g., summaries of the paragraphs) for each paragraph. The extractions 872 are combined (e.g., concatenated) in step 874 and provided to a large language model 876 along with the rewritten query 866. The enterprise generative artificial intelligence system can use the combined extractions 874 to generate root causes 882 for determining a final response 878. For example, the large language model 876 can generate a final response 878 based on the root causes 882 and / or present the root causes 882 (or a summary of the root causes) along with the final response 878 (e.g., for citation or attribution purposes).

[0161] In some embodiments, the extraction step is optional, and instead of extracting or combining extractions, the paragraphs can be concatenated and provided to a large language model 876. The large language model 844 can generate a final response 878 based on the rewritten query 866 and the combined extraction 874. In some embodiments, the large language model 876 can post-process the final result 878 before providing it to the user (e.g., using an orchestrator). For example, it can be translated based on the viewpoint, formatted, include references and attributes, etc.

[0162] Figure 9 Flowchart 900 depicts an example iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps can be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been removed but included for clarity of illustration.

[0163] exist Figure 9 In the example of , an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) includes one or more retrieval modules 904 and one or more understanding modules 906. For example, the retrieval module 904 may include one or more large language models, and the understanding module may include one or more other large language models. These modules and the iterative interaction (e.g., communication) between these modules can allow the enterprise generative artificial intelligence system to achieve the technical features and technical benefits discussed herein.

[0164] exist Figure 9 In the example of , an enterprise generative artificial intelligence system may receive initial input 902 from a user or another system. For example, an orchestrator module 903 may receive input 902. The enterprise generative artificial intelligence system may provide this input to a retrieval module 904 (e.g., corresponding to one or more of agents 506 and / or tools 508), which may then extract and "retrieve" information from embedding storage 908. For example, retrieval module 904 may retrieve passages based on the input by using one or more similarity heuristics (e.g., an ANN algorithm) executed on embedding storage 908 (e.g., one or more vector stores) to retrieve passages (or data records) related to the input.

[0165] The enterprise generative artificial intelligence system can use this retrieved information to generate an initial prompt for the understanding module 906. The understanding module 906 can process the initial prompt and determine whether it has enough information to meet the criteria (e.g., answer the question) based on the initial input. See, for example, step 907. If it has enough information to meet the initial input, the understanding module can then provide the results to a recipient (e.g., see, for example, step 913), such as the user or system that provided the initial input. However, if the understanding module 906 determines that it does not have enough information to meet the criteria based on the initial input, it can further synthesize information through an iterative process that provides the core benefits of the system.

[0166] There are many reasons why understanding module 906 may need additional information. For example, traditional systems use a single-pass process that only processes a portion of a complex input. Enterprise generative artificial intelligence systems address this problem by triggering subsequent iterations to solve for the remaining portion of the complex input and including context to further refine the process.

[0167] More specifically, if the understanding module 906 determines that it needs additional information to satisfy the initial input, it can generate context-specific data (or simply "context") that will inform future iterations of the process and help the system satisfy the initial input more efficiently and accurately. The context is based on the root cause used by the understanding module 906 when it is processing the query (or other input). For example, the understanding module 906 can receive segments of information retrieved by the retrieval module 904. The segments can be, for example, paragraphs of (one or more) data records, and the segments can be associated with embeddings from the embedding data store 908 that facilitate the processing of the understanding module 906. The query and root cause generator 912 of the understanding module 906 can process this information and generate a root cause for why the result produced by the generator 912 was produced. The root cause can be stored by the enterprise generative artificial intelligence system in the historical root cause data store 910 and provide a basis for subsequent iterations of the context.

[0168] More specifically, subsequent iterations may include the understanding module 906 generating a new query, request, or other output, which is then passed back to the retrieval module. The retrieval module 904 may process the new query and retrieve additional information. The system then generates a new prompt based on the additional information and context. The understanding module 906 may process the new prompt and again determine whether it requires additional information. If it requires additional information, the enterprise generative artificial intelligence system may repeat (e.g., iterate) the process until the understanding module 906 can meet the criteria based on the initial input, at which point the understanding module 906 may generate an output 914 (e.g., "answer" or "I don't know"). For example, if no relevant paragraphs are generated or retrieved (e.g., by applying the rules of the understanding module 906) and / or not enough relevant paragraphs are generated, retrieved, and / or extracted, then the answer "I don't know" is generated. The understanding module 906 can prevent hallucinations and improve the performance of "I don't know" queries while saving calls to a model (e.g., a large language model).

[0169] In some embodiments, whether sufficient information exists can be determined and / or correlated based on the number of retrieved paragraphs, but relevant information has not been extracted (e.g., by comprehension module 906). For example, a threshold number or percentage of retrieved paragraphs (e.g., a particular number or percentage of retrieved paragraphs) may need to be met for relevant information to be extracted for business comprehension module 906 to determine that it has sufficient information to answer the query. In another example, a threshold number or percentage of retrieved paragraphs (e.g., 4 paragraphs or 80% of the retrieved paragraphs) that do not extract relevant information may cause business comprehension module 906 to determine that it does not have sufficient information to answer the query.

[0170] The enterprise generative artificial intelligence system may also implement supervisory functions, such as stopping conditions that prevent the system from hallucinating or otherwise providing incorrect answers. Stopping conditions may also prevent the system from executing an infinite iterative loop. In one example, the enterprise generative artificial intelligence system may limit the number of iterations that can be performed before the understanding module 906 will provide an output result or indicate that an output result cannot be found. Users may also receive feedback 916 that may be stored in a feedback data store 918. In some embodiments, the enterprise generative artificial intelligence system may use feedback to improve the accuracy and / or reliability of the system. As discussed elsewhere herein, it will be understood that in some embodiments, the functionality of the understanding module may be included within the orchestrator.

[0171] Figure 10Flowchart 1000 depicts an example generative artificial intelligence process using unstructured and structured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps can be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been removed but included for clarity of illustration.

[0172] In step 1002, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) processes input (e.g., complex input) using an orchestrator (e.g., orchestrator module 504). The orchestrator may include one or more first large language models, first machine learning models, and / or first functions (e.g., supervised functions).

[0173] In step 1004, the enterprise generative artificial intelligence system utilizes an orchestrator to select a first agent from a plurality of different agents based on the processed input. In some embodiments, the first agent (e.g., an unstructured data retrieval agent) is generated by a retrieval agent module (e.g., retrieval agent module 506-1) and / or an unstructured data retriever agent module 506-2. In some embodiments, the first agent includes one or more second large language models (e.g., a large language model different from the first large language model).

[0174] In step 1006, the enterprise generative artificial intelligence system uses a first agent to retrieve one or more first data records from an unstructured data set. The first data record can be an unstructured data record retrieved from one or more vector data stores and / or based on one or more vector data stores. For example, the first agent can implement Figure 9 In some embodiments, the first agent uses (e.g., executes, accesses, and / or controls) one or more tools (e.g., unstructured data retrieval tool module 508-1) to retrieve unstructured data records. In some embodiments, the first agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process (e.g., Figure 8A and Figure 9 ) and / or non-iterative generative AI processes (e.g., as Figure 7 and Figures 8B to 8C) to retrieve one or more first data records from an unstructured dataset. In one example, the first agent instructs one or more unstructured data retrieval tools based on embeddings in the vector storage. For example, the first agent and / or tool can use an artificial intelligence-based similarity search (e.g., an ANN algorithm) to identify and retrieve paragraphs with similar embedding values ​​(e.g., closest in vector space based on one or more thresholds).

[0175] In step 1008, the enterprise generative artificial intelligence system utilizes the orchestrator to select a second agent from the plurality of different agents based on the processed input. In some embodiments, the second agent (e.g., an unstructured data retrieval agent) is generated by a retrieval agent module (e.g., retrieval agent module 506-1) and / or a structured data retriever agent module 506-3. In some embodiments, the second agent includes one or more third large language models (e.g., a large language model different from the first large language model and / or the second large language model).

[0176] In step 1010, the enterprise generative artificial intelligence system uses a second agent to retrieve one or more second data records from the structured data set. In some embodiments, the second agent uses (e.g., executes, accesses, and / or controls) one or more other tools (e.g., structured data retrieval tool module 508-2) to retrieve the structured data records. In some embodiments, the second agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from the structured data set (e.g., Figure 7 In some embodiments, the second agent instructs one or more structured data retrieval tools based on a data model that describes one or more types of relationships of the data model.

[0177] In step 1012, the enterprise generative artificial intelligence system utilizes an orchestrator to generate a natural language summary of one or more first data records and one or more second data records. In some embodiments, the orchestrator's large-scale language model generates the natural language summaries based on the natural language summaries generated by the first agent and the second agent. For example, the first agent may generate a natural language summary of a retrieved first data record, and the second agent may generate another natural language summary of a retrieved second data record.

[0178] In step 1014, the enterprise generative artificial intelligence system transmits a natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the input. In some embodiments, a communication module (e.g., communication module 530) transmits the natural language summary to the user and / or system that provided or generated the input.

[0179] In some embodiments, the orchestrator parses the input into different parts (e.g., segments) and routes each part to a corresponding agent. For example, the orchestrator may determine that a first segment requires an unstructured data retrieval operation and route the first segment to a first agent. Similarly, the orchestrator may determine that a second segment requires a structured data retrieval operation and route the second segment to a second agent. In some embodiments, the orchestrator may launch agents to process the input and / or orchestrate agents for parallel execution as needed.

[0180] Figure 11 A flowchart 1100 is depicted illustrating an example generative artificial intelligence process using unstructured and structured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps may be repeated, reorganized for parallel execution, and / or reordered. Furthermore, for clarity, some steps may have been removed to avoid providing excessive information, and some steps may have been included but could have been removed for clarity.

[0181] In step 1102, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) obtains a query. In some embodiments, an interface module (e.g., interface module 528) and / or an orchestrator module (e.g., orchestrator module 504) obtains the query (e.g., from a user and / or the system).

[0182] In step 1104, the enterprise generative artificial intelligence system interprets the query. Although a query is used in this example, it will be understood that other types of input can be used instead. In some embodiments, the orchestrator module interprets the query.

[0183] In step 1106, the enterprise generative artificial intelligence system selects a first agent from among the plurality of different agents based on the interpretation of the query. In some embodiments, the orchestrator module and / or another agent (e.g., unstructured data retriever agent module 506-1) selects the first agent (e.g., unstructured data retriever agent module 506-2).

[0184] In step 1108, the enterprise generative artificial intelligence system uses the first agent to retrieve one or more first data records from the unstructured data set. The first data record can be an unstructured data record retrieved from one or more vector data stores and / or based on one or more vector data stores. In some embodiments, the first agent uses (e.g., executes, accesses and / or controls) one or more tools (e.g., unstructured data retrieval tool module 508-1) to retrieve the unstructured data record. In some embodiments, the first agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process (e.g., Figure 8A and Figure 9 ) and / or non-iterative generative AI processes (e.g., as Figures 8B to 8C ) to retrieve one or more first data records from an unstructured dataset. In one example, the first agent instructs one or more unstructured data retrieval tools based on the embeddings in the vector store. For example, the first agent and / or tool can use an AI-based similarity search to identify and retrieve paragraphs with similar embedding values.

[0185] In step 1110, the enterprise generative artificial intelligence system selects a second agent from the plurality of different agents based on the interpretation of the query. In some embodiments, the orchestrator module and / or another agent (e.g., unstructured data retriever agent module 506-1) selects the second agent (e.g., unstructured data retriever agent module 506-2).

[0186] In step 1112, the enterprise generative artificial intelligence system uses a second agent to retrieve one or more second data records from the structured data set. In some embodiments, the second agent uses (e.g., executes, accesses, and / or controls) one or more other tools (e.g., structured data retrieval tool module 508-2) to retrieve the structured data records. In some embodiments, the second agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from the structured data set (e.g., Figure 7 In some embodiments, the second agent instructs one or more structured data retrieval tools based on a data model that describes one or more types of relationships of the data model.

[0187] In step 1114, the enterprise generative artificial intelligence system generates a first natural language summary of one or more first data records. In some embodiments, the first agent generates the first natural language summary. In step 1116, the enterprise generative artificial intelligence system generates a visualization based on one or more second data records from the structured dataset. In some embodiments, a visualization agent module (e.g., visualization agent module 506-9) generates the visualization. For example, the visualization agent module may execute a visualization tool (e.g., visualization tool module 508-7) to generate the visualization. In step 1118, the enterprise generative artificial intelligence system generates a second natural language summary based on the first natural language summary and the visualization. In some embodiments, the orchestrator module generates the second natural language summary. In step 1120, the enterprise generative artificial intelligence system transmits the second natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the query. In some embodiments, a communication module (e.g., communication module 530) transmits the second natural language summary.

[0188] Figure 12 Flowchart 1200 depicts an example of a non-iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps can be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been included but could have been removed for clarity.

[0189] In step 1202, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) receives input (e.g., complex input). In some embodiments, an orchestrator module (e.g., orchestrator module 504) receives the input. In step 1204, the enterprise generative artificial intelligence system retrieves a plurality of unstructured data records from a data store based on the input. In some embodiments, the orchestrator instructs an unstructured data retriever agent module (e.g., unstructured data retriever agent module 506-2) to retrieve the unstructured data records, and the agent instructs an unstructured data retrieval tool (e.g., unstructured data retrieval tool 508-1) to perform a retrieval function. In step 1206, the enterprise generative artificial intelligence system filters the plurality of unstructured data records. In some embodiments, the unstructured data retriever agent module instructs a filtering tool (e.g., filter tool module 508-9) to filter the unstructured data records.

[0190] In step 1208, the enterprise generative artificial intelligence system identifies key points of the unstructured data records. In some embodiments, the unstructured data retriever agent module identifies the key points. In step 1210, the enterprise generative artificial intelligence system generates a summary of the filtered data records based on the identified key points. In some embodiments, an understanding module (e.g., understanding module 510) generates the summary. For example, the understanding module may include a large language model that takes the key points as input and generates a natural language summary based on the key points. As described elsewhere herein, the understanding module may be a component of the orchestrator.

[0191] In step 1212, the enterprise generative artificial intelligence system generates a result based on the summary. The result can be the summary itself or the output of another large language model. In some embodiments, the summary can be provided to another large language model (e.g., "answering" the large language model) to provide a contextualized final result. More specifically, the other large language model can generate results based on the user who submitted the initial query (e.g., based on the user role). In step 1214, the enterprise generative artificial intelligence system provides the result to the recipient associated with the input (e.g., the user or system that originally provided the input). In some embodiments, the communication module (e.g., communication module 530) provides the result via a communication network (e.g., communication network 408).

[0192] Figure 13Flowchart 1300 depicts an example of a generative artificial intelligence process using structured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that some or all of the steps can be repeated, reorganized for parallel execution, and / or reordered, where applicable. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been removed but included for clarity of illustration.

[0193] In step 1302, the enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) accesses a data model. The data model can include various types, and each type can describe different data fields, operations, and functions. In some embodiments, each type represents a different object (e.g., a machine in a factory), and each type can include context that enables another model (e.g., a large language model) to process information and / or generate results more accurately and / or efficiently. In some embodiments, an orchestrator (e.g., orchestrator module 504), an agent (e.g., agent 506), and / or a tool (e.g., tool module 508) accesses the data model. In a specific implementation, a type system retriever agent (e.g., type system retriever agent module 506-4) accesses the data model.

[0194] In step 1304, the enterprise generative artificial intelligence system receives input (e.g., complex input). In some embodiments, the orchestrator module receives the input. In step 1306, the enterprise generative artificial intelligence system determines a plurality of keywords based on the input. In some embodiments, the orchestrator module determines the keywords.

[0195] In step 1308, the enterprise generative artificial intelligence system retrieves a subset of the data model based on a similarity search on the data model using multiple keywords. In some embodiments, a type system agent (e.g., type system retriever agent module 506-4) performs a similarity search and retrieves the subset of the data model. In step 1310, the enterprise generative artificial intelligence system retrieves structured data from one or more structured data stores based on the subset of the data model. In some embodiments, a structured data retriever agent (e.g., structured data retriever agent module 506-3) retrieves the structured data.

[0196] In step 1312, the enterprise generative artificial intelligence system determines, via the large language model, an output of the large language model based on the input and the corresponding large language model context of the type of the data model subset. In step 1314, the enterprise generative artificial intelligence system transmits the output of the large language model to a recipient associated with the input. In some embodiments, a communication module (e.g., communication module 530) transmits the output to the user and / or system that provided the initial input.

[0197] Figure 14 Flowchart 1400 depicts an example operation of an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. In this flowchart and / or sequence diagram and other flowcharts and / or sequence diagrams, the flowchart illustrates a sequence of steps by way of example. It should be understood that some or all of the steps may be repeated, reorganized for parallel execution, and / or reordered, where applicable. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been removed but included for clarity of illustration.

[0198] In step 1402, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) obtains input (e.g., complex input). In some embodiments, an interface module (e.g., interface module 528) receives the input. In step 1404, the enterprise generative artificial intelligence system provides the input to an orchestrator (e.g., orchestrator module 504). The orchestrator may include one or more first large language models and / or one or more machine learning models. In some embodiments, the interface module provides the input to the orchestrator.

[0199] In step 1406, the enterprise generative artificial intelligence system utilizes the orchestrator to select inputs for the selected one or more agents (e.g., agent 506). In step 1408, the enterprise generative artificial intelligence system utilizes the orchestrator to route the inputs to the selected one or more agents.

[0200] In step 1410, the enterprise generative artificial intelligence system utilizes one or more corresponding tools associated with the selected one or more agents to generate corresponding outputs for each of the one or more tools. In some embodiments, the corresponding tools associated with the first agent include one or more retriever models, the corresponding tools associated with the second agent include a database query generator, the corresponding tools associated with the third agent include an API generator, and the corresponding retrieval tool associated with the fourth agent includes an iterative context-based generative artificial intelligence tool. The first portion of the input is routed to the first agent, the second portion of the input is routed to the second agent, and the third portion of the input is routed to the third agent. The first agent can be configured to generate one or more document retrieval requests, the second agent is configured to generate one or more SQL queries, the third agent is configured to generate one or more API calls for one or more artificial intelligence applications, and the fourth agent is configured to trigger the enterprise generative artificial intelligence system. The first agent, the second agent, and / or the third agent can execute in parallel with each other based on control instructions provided by the orchestrator.

[0201] In step 1412, the enterprise generative AI system transforms each of the one or more corresponding outputs into a natural language format. In some embodiments, the selected one or more agents transform the outputs. In step 1414, the enterprise generative AI system inputs the transformed outputs into a comprehension module. The comprehension module may include one or more third language models. The third large language model may be the same as or different from the first large language model. In step 1416, the enterprise generative AI system provides the outputs of the comprehension module to a recipient associated with the input.

[0202] Figure 15 Figure 1500 depicts an example of a computing device 1502. Any of the systems, engines, data stores, and / or networks described herein may include one or more instances of computing device 1502. In some embodiments, the functionality of computing device 1502 is modified to perform some or all of the functionality described herein. Computing device 1502 includes a processor 1504, memory 1506, storage 1508, input device 1510, a communication network interface 1512, and an output device 1514 communicatively coupled to a communication channel 1516. Processor 1504 is configured to execute executable instructions (e.g., a program). In some embodiments, processor 1504 includes circuitry or any processor capable of processing executable instructions.

[0203] Memory 1506 stores data. Some examples of memory 1506 include storage devices such as RAM, ROM, RAM cache, virtual memory, etc. In various embodiments, working data is stored in memory 1506. Data in memory 1506 can be cleared or eventually transferred to storage 1508.

[0204] Storage 1508 includes any storage configured to retrieve and store data. Some examples of storage 1508 include a flash drive, a hard drive, an optical drive, cloud storage, and / or magnetic tape. Each of memory system 1506 and storage system 1508 includes a computer-readable medium that stores instructions or programs executable by processor 1504.

[0205] Input device 1510 is any device for inputting data (e.g., a mouse and keyboard). Output device 1514 outputs data (e.g., a speaker or display). It will be understood that storage 1508, input device 1510, and output device 1514 may be optional. For example, a router / switch may include processor 1504 and memory 1506 as well as devices for receiving and outputting data (e.g., communication network interface 1512 and / or output device 1514).

[0206] The communication network interface 1512 can be coupled to a network (e.g., network 408) via link 1518. The communication network interface 1512 can support communication via an Ethernet connection, a serial connection, a parallel connection, and / or an ATA connection. The communication network interface 1512 can also support wireless communications (e.g., 802.11 a / b / g / n, WiMax, LTE, Wi-Fi). It will be apparent that the communication network interface 1512 can support many wired and wireless standards.

[0207] It will be understood that the hardware elements of computing device 1502 are not limited to Figure 15 Computing device 1502 may include more or fewer hardware, software, and / or firmware components than those depicted (e.g., drivers, operating system, touch screen, and / or biometric analyzer, etc.). Furthermore, hardware elements may share functionality and still be within the various embodiments described herein. In one example, encoding and / or decoding may be performed by processor 1504 and / or a coprocessor located on a GPU (i.e., NVidia).

[0208] Example types of computing devices and / or processing devices include one or more microprocessors, microcontrollers, reduced instruction set computers (RISCs), complex instruction set computers (CISCs), graphics processing units (GPUs), data processing units (DPUs), virtual processing units, associative processing units (APUs), tensor processing units (TPUs), vision processing units (VPUs), neuromorphic chips, AI chips, quantum processing units (QPUs), brain system wafer-level engines (WSEs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or discrete circuit systems.

[0209] It will be understood that an “engine,” “system,” “data store,” and / or “database” may include software, hardware, firmware, and / or circuitry. In one example, one or more software programs comprising instructions executable by a processor may perform one or more of the functions of the engine, data store, database, or system described herein. In another example, the circuitry may perform the same or similar functions. Alternative embodiments may include more, fewer, or functionally equivalent engines, systems, data stores, or databases and still be within the scope of the present embodiments. For example, the functionality of the various systems, engines, data stores, and / or databases may be combined or divided differently. The data store or database may include cloud storage. It will also be understood that the term “or,” as used herein, may be interpreted in an inclusive or exclusive sense. In addition, multiple instances may be provided for a resource, operation, or structure described herein as a single instance.

[0210] The data store described herein can be any suitable structure (e.g., an active database, a relational database, a self-referential database, a table, a matrix, an array, a flat file, a document-oriented storage system, and a non-relational No-SQL system, etc.), and can be cloud-based or otherwise. The systems, methods, engines, data stores, and / or databases described herein can be at least partially processor-implemented, wherein one or more specific processors are examples of hardware. For example, at least some of the operations of the method can be performed by one or more processors or processor-implemented engines. In addition, one or more processors can also operate to support the performance of related operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations can be performed by a computer group (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application program interfaces (APIs)).

[0211] The execution of certain operations may be distributed among processors, not only residing within a single machine, but also deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0212] Throughout the specification, multiple instances can be implemented as components, operations or structures described as single instances. Although the individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations can be performed simultaneously and do not need to be operated in the order shown. The structure and functionality presented as separate components in the example configuration can be implemented as combined structures or components. Similarly, the structure and functionality presented as separate components can be implemented as separate components. These and other variations, modifications, additions and improvements fall within the scope of this paper's theme.

[0213] The claimed solution, rooted in computer technology, overcomes problems particularly arising in the field of computer technology. Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to process input using an orchestrator. Selecting, using the orchestrator, a first agent from a plurality of different agents based on the processed input. Retrieving, using the first agent, one or more first data records from an unstructured data set. Selecting, using the orchestrator, a second agent from a plurality of different agents based on the processed input. Retrieving, using the second agent, one or more second data records from a structured data set. Generating, using the orchestrator, a natural language summary of the one or more first data records and the one or more second data records, and transmitting the natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the input.

[0214] In some embodiments, the orchestrator includes one or more first large-scale language models. In some embodiments, the first agent includes one or more second large-scale language models, and the second agent includes one or more third large-scale language models. The first agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve one or more first data records from an unstructured dataset. The second agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from a structured dataset. The first agent instructs the one or more unstructured data retrieval tools based on embeddings in the vector storage. Selecting, using the orchestrator, a first agent from among the plurality of different agents based on the processed input further includes generating a second input based on the first portion of the input and routing the second input to the first agent. Selecting, using the orchestrator, a second agent from among the plurality of different agents based on the processed input further includes generating a third input based on the second portion of the input and routing the third input to the second agent. The first agent and the second agent execute in parallel and perform their respective searches in parallel.

[0215] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: obtain a query; interpret the query; select a first agent from a plurality of different agents based on the interpretation of the query; retrieve one or more first data records from an unstructured dataset using the first agent; select a second agent from a plurality of different agents based on the interpretation of the query; retrieve one or more second data records from a structured dataset using the second agent; generate a first natural language summary of the one or more first data records; generate a visualization based on the one or more second data records from the structured dataset; generate a second natural language summary based on the first natural language summary and the visualization; and transmit the second natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the query.

[0216] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: receive input; retrieve a plurality of unstructured data records from a data store based on the input; filter the plurality of unstructured data records; identify key points of the filtered data records; generate a summary of the filtered data records based on the identified key points; generate a result based on the summary; and provide the result to a recipient associated with the input.

[0217] In some embodiments, systems, methods, and non-transitory computer-readable media are configured to: determine a number of identified data records; compare the number to a threshold; and skip key point identification and generate a summary based on the filtered data records instead of the key points.

[0218] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: access a data model, wherein the data model includes a plurality of types, and wherein each type describes one or more corresponding data fields, operations, and functions, and wherein each type represents a different object, and wherein each type includes a corresponding large language model context; receive input; determine a plurality of keywords based on the input; retrieve a subset of the data model based on a similarity search on the data model using the plurality of keywords; retrieve structured data from one or more structured data stores based on the subset of the data model; determine, using the large language model, an output of the large language model based on the input and corresponding large language model contexts of types of the subset of the data model; and transmit the output of the large language model to a recipient associated with the input.

[0219] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: obtain input; provide the input to an orchestrator, the orchestrator comprising one or more first large language models and one or more machine learning models; select, by the orchestrator, one or more agents from a plurality of agents, each agent comprising a corresponding second large language model and configured to access one or more corresponding tools from a plurality of different tools; route the input to the selected one or more agents using the orchestrator; generate corresponding outputs for each of the one or more tools using one or more corresponding tools associated with the selected one or more agents; transform each of the one or more corresponding outputs into a natural language format; input the transformed outputs into a comprehension module, the comprehension module comprising one or more third language models; and provide the outputs of the comprehension module to a recipient associated with the first input.

[0220] The orchestrator includes one or more second large language models. In some embodiments, the corresponding tool associated with the first agent includes one or more retriever models, the corresponding tool associated with the second agent includes a database query generator, the corresponding tool associated with the third agent includes an API generator, and the corresponding retrieval tool associated with the fourth agent includes an iterative context-based generative artificial intelligence tool. The first part of the input is routed to the first agent, the second part of the input is routed to the second agent, and the third part of the input is routed to the third agent. The first agent is configured to generate one or more document retrieval requests, the second agent is configured to generate one or more SQL queries, the third agent is configured to generate one or more API calls for one or more artificial intelligence applications, and the fourth agent is configured to trigger the enterprise generative artificial intelligence system. The first agent, the second agent, and the third agent execute in parallel with each other based on control instructions provided by the orchestrator. In some embodiments, the agents include one or more machine learning models. The above describes (one or more) present inventions with reference to example embodiments. It will be apparent to those skilled in the art that various modifications may be made and other embodiments may be used without departing from the broader scope of the present invention(s). Therefore, the present invention(s) is intended to cover these and other changes to the exemplary embodiments.

[0221] According to an example disclosed herein, a computer-implemented method is provided, comprising: utilizing an orchestrator to manage multiple agents to generate responses to inputs, wherein the orchestrator employs one or more large language models to process or deconstruct a prompt into a series of instructions for different agents, wherein each agent employs one or more machine learning models to process different inputs or different portions of the inputs associated with the prompts; utilizing the orchestrator to indicate a retrieval request related to the input to one or more of the multiple agents; receiving data from multiple data domains from one or more of the multiple agents based on instructions from the orchestrator; utilizing the orchestrator to analyze the retrieved data to formulate one or more responses to the prompts, wherein the orchestrator provides additional retrieval requests to the one or more agents to retrieve additional data that satisfies contextual validation criteria associated with the inputs; utilizing the orchestrator to output at least one of the following: a validated response from the one or more responses to the inputs that satisfies the contextual validation criteria; and a portion of the data retrieved by the one or more agents that is related to the inputs.

[0222] The additional retrieval request may include a request for additional data (which may be different from the retrieved data) that may be used to check, validate, and / or verify the retrieved data and / or one or more responses. For example, the additional retrieval request may include a request configured to retrieve data similar to the retrieved data but from one or more different data sources. The additional data may include data from a data domain (and / or a different portion of the same data domain) that is different from the data domain (and / or a corresponding portion of the data domain) from which the retrieved data was retrieved. Thus, the additional data may provide alternative data to the retrieved data based on which the retrieved data and / or one or more responses may be checked, validated, and / or verified.

[0223] Additional retrieval requests may be based on one or more responses to a prompt. For example, additional retrieval requests may be formulated to check or validate one or more responses to a prompt. Additional retrieval requests may, for example, include one or more queries formulated based on one or more responses to a prompt to query the content of the one or more responses to a prompt. The method may include indicating portions of data records and their corresponding data sources used to generate the retrieved data (and / or additional data) and / or the one or more responses to the prompt. Portions of data records and their corresponding data sources may be indicated as a portion of the retrieved data and / or a portion of the additional data. Additional retrieval requests may include requests to inspect data records (and / or their respective data sources) to check, validate, or verify one or more responses to a prompt.

[0224] The method may comprise validating one or more responses to the prompt. The validating may comprise comparing the one or more responses to the prompt with the additional data. For example, the method may comprise determining a measure of similarity and / or consistency between the one or more responses to the prompt and the additional data. The measure of similarity and / or consistency may be a numerical measure suitable for comparison with a threshold value. The validating may comprise validating the response to the prompt if the measure of similarity and / or consistency (between the response to the prompt and the additional data) is greater than a first threshold value. The validating may comprise not validating the response to the prompt if the measure of similarity and / or consistency (between the response to the prompt and the additional data) is not greater than a second threshold value (which may be the same as the first threshold value).

[0225] The request and receipt of additional data and / or verification of one or more responses to a prompt provides a synergistic technical effect of improving the integrity and reliability of the output provided in response to the prompt. These features are based on a technical understanding of the internal functionality of a model (e.g., a large language model) and / or the functions used to generate responses. For example, it has been recognized that such models and / or functions can be based on probabilistic methods that may, for example, generate some inconsistent or inaccurate results. Requesting and receiving additional data and / or verifying responses can avoid or mitigate the output of inconsistent or inaccurate results. Providing input-related portions of data retrieved by one or more agents in addition to providing verified responses can allow further confirmation of the results (e.g., by a user receiving the output), thereby further improving the integrity and accuracy of the results.

[0226] Processing or deconstructing the prompt into a series of instructions for different agents may include determining instructions for agents that depend on the nature of the agents. For example, the agents may include different agents for performing different functions (e.g., retrieving unstructured data, receiving structured data, etc.). The different agents may be configured to receive instructions in different formats and / or languages ​​(e.g., different programming languages). The instructions determined for the different agents may depend on the functions the agents are configured to perform and / or the formats and / or languages ​​the agents are configured to receive. For example, a first instruction may be determined for a first agent, wherein the first instruction is determined according to a format and / or language to be received and processed by the first agent (e.g., according to the format and / or language of the instructions that the first agent is configured to receive). A second instruction may be determined for a second agent, wherein the second instruction is determined according to a format and / or language to be received and processed by the second agent (e.g., according to the format and / or language of the instructions that the second agent is configured to receive). The format and / or language of the instructions determined for the first agent may be different from the format and / or language of the instructions determined for the second agent. One or more large language models employed by the orchestrator can be configured through training to determine instructions for an agent based on the format and / or language of instructions that the agent is configured to receive. That is, the large language model can be configured through training to output instructions in different formats and / or languages ​​for different agents. Model training can be implemented continuously, asynchronously, and using feedback (e.g., reinforcement learning, etc.). In some illustrative examples, the orchestrator can transform a first portion of the input into, for example, a SQL query and send it to an unstructured data retriever agent, and / or transform a second portion of the input into an API call and send it to an API agent.

[0227] At least one of the one or more agents can be configured to determine instructions for one or more tools to perform one or more operations (e.g., such as structured data retrieval and / or unstructured data retrieval, etc.). Different tools can be configured to receive instructions in different formats and / or languages ​​(e.g., different programming languages). The instructions determined for different tools can depend on the functions or operations that the tools are configured to perform and / or the formats and / or languages ​​that the tools are configured to receive. For example, a first instruction can be determined for a first tool (e.g., by a first agent), wherein the first instruction is determined based on the format and / or language to be received and processed by the first tool (e.g., based on the format and / or language of the instructions that the first tool is configured to receive). A second instruction can be determined for a second tool (e.g., by the first agent or by a second agent), wherein the second instruction is determined based on the format and / or language to be received and processed by the second tool (e.g., based on the format and / or language of the instructions that the second tool is configured to receive). The format and / or language of instructions determined for a first tool may be different than the format and / or language of instructions determined for a second tool. One or more machine learning models employed by one or more agents may be configured through training to determine instructions for a tool based on the format and / or language of instructions that the tool is configured to receive. That is, the machine learning model may be configured through training to output instructions in a given format and / or language for a given tool, and / or may be configured through training to output instructions in different formats and / or languages ​​for different tools.

[0228] Method can comprise: provide instruction and / or inquiry to code generation intelligent agent and / or tool.Code generation intelligent agent and / or tool can be configured to generate source code, machine code and / or other computer code.In some examples, code generation intelligent agent can be configured to instruct code generation tool to generate source code, machine code and / or other computer code.Code generation intelligent agent can be configured to determine what code is needed and / or should be generated for what intelligent agent, tool and / or other entity or application.Code generation intelligent agent can instruct code generation tool to generate the function that is suitable for receiving code and / or receive the language and / or format of intelligent agent, tool and / or other entity or application.Code generation intelligent agent and / or tool can output the code of the format and / or language that is configured to be received by specific intelligent agent, tool and / or other entity or application.

[0229] The method may include receiving data and converting the data into a natural language format to provide to a multimodal model (e.g., a large language model). For example, one or more agents may receive data and convert the data into a natural language format to provide to a multimodal model (e.g., a large language model) employed by an orchestrator. In some examples, one or more agents may receive data and summarize the content of the data in a natural language format to provide to a multimodal model (e.g., a large language model).

[0230] In various examples contemplated herein, an arranger, an agent and / or a tool can be configured to receive an input and generate an output based on the input. The output can be provided to another arranger, an agent and / or a tool for further processing. The output provided to another arranger, an agent and / or a tool can be configured in a format and / or language that can be read and processed by another arranger, an agent and / or a tool. An arranger, an agent and / or a tool (and / or a model adopted by an arranger, an agent and / or a tool) can be configured to determine the output of a format and / or language that can be read and processed by another arranger, an agent and / or a tool through training. By providing the output of a language and / or format to be read by another arranger, an agent and / or a tool, an arranger, an agent and / or a tool can collaborate with each other and interoperate to provide an overall output (e.g., in the form of at least one of the following: a verified response in one or more responses to an input that satisfies a contextual validation criterion; and a portion of the data retrieved by one or more agents that is relevant to the input). Using an orchestrator, one or more agents and / or one or more tools can allow different orchestrators, agents, and / or tools to perform different operations required to produce an overall output. This can allow different orchestrators, agents, and / or tools to be configured (e.g., through training) to perform different operations or functions. Different orchestrators, agents, and / or tools can perform different operations or functions in parallel. This can be used to improve the computational efficiency of providing outputs in response to inputs.

[0231] Outputting at least one of a validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input may include displaying a representation of at least one of the validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input on an electronic display. Outputting at least one of a validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input may include transmitting at least one of the validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input to a computing device for display at the computing device.

[0232] At least one of the validated responses in one or more responses to an input that satisfies contextual validation criteria and the portion of the data retrieved by the one or more agents that is related to the outputted input depends on the technical functionality of one or more multimodal models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more multimodal models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more multimodal models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more multimodal models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more multimodal models and / or the one or more machine learning models may include simulation of the one or more multimodal models and / or the one or more machine learning models using software.

[0233] The orchestrator may generate intermediate instructions to multiple agents, wherein the intermediate instructions include at least one of a portion of an input, a question related to the input generated by one or more multimodal models, and a follow-up question related to the answer generated by the one or more multimodal models. Outputting the portion of the data retrieved by the one or more agents related to the input may include a source reference to at least a portion of a verified response. Contextual validation criteria may include a threshold for identifying source material from an enterprise data system used to validate the response. Managing multiple agents may include iterative processing or multiple instructions from the orchestrator. Retrieving data from multiple data domains may include time series data, structured data, and unstructured data. At least one of the one or more agents may instantiate a tool to operate on the instructions, retrieved data, or intermediate data. The operation may include at least one of calculation, translation, formatting, and visualization. One or more agents may be trained on specific machine learning models for different domains. At least one agent may employ a type system to unify incompatible data from different data sources.

[0234] According to an example disclosed herein, a computer-implemented method is provided, comprising: utilizing an orchestrator to select one or more agents from a plurality of different agents based on processed input; utilizing the selected one or more agents to retrieve data records from an unstructured dataset and additional data records from a structured dataset; utilizing the orchestrator to generate a natural language summary of the data records from the unstructured dataset and the additional data records from the structured dataset; and transmitting the natural language summary of the data records and the additional data records as a response to the input.

[0235] The selected agent from a plurality of different agents may include a first agent configured to retrieve data records from an unstructured dataset and a second agent configured to retrieve additional data records from a structured dataset. The data records may be retrieved from the unstructured dataset by the first agent. The additional data records may be retrieved from the structured dataset by the second agent. The first agent may instruct one or more unstructured data retrieval tools to retrieve data records from the structured dataset. The first agent may be configured to generate instructions for the one or more unstructured data retrieval tools. The first agent may generate instructions in a format and / or language that the one or more unstructured data retrieval tools are configured to receive. The second agent may instruct one or more structured data retrieval tools to retrieve additional data records from the structured dataset. The second agent may be configured to generate instructions for the one or more structured data retrieval tools. The second agent may generate instructions in a format and / or language that the one or more structured data retrieval tools are configured to receive. The first agent and / or the second agent may employ (e.g., through training) one or more machine learning models configured to generate instructions for one or more data retrieval tools. The one or more machine learning models employed by the one or more agents may be configured through training to determine instructions for the tool based on the format and / or language of the instructions that the tool is configured to receive. That is, the machine learning model may be configured through training to output instructions in a given format and / or language for a given tool, and / or may be configured through training to output instructions in different formats and / or languages ​​for different tools. The machine learning model may include some or all of the different types or modalities of models described herein (e.g., multimodal machine learning models, large language models, data models, statistical models, audio models, visual models, audio-visual models, etc.).

[0236] The method may include providing instructions and / or inquiries to a code-generating agent and / or tool. The code-generating agent and / or tool may be configured to generate source code, machine code and / or other computer codes. In some examples, the code-generating agent may be configured to indicate that a code-generating tool generates source code, machine code and / or other computer codes. The code-generating agent may be configured to determine what code is needed and / or should be generated for what agent, tool and / or other entity or application. The code-generating agent may indicate that a code-generating tool generates the function that is suitable for receiving code and / or the language and / or format of receiving agent, tool and / or other entity or application. The code-generating agent and / or tool may output the code configured to the format and / or language received by a specific agent, tool and / or other entity or application.

[0237] The method may include receiving data and converting or packaging the data into a compatible format for provision to a multimodal model. For example, one or more agents may receive data and convert the data into a natural language format for provision to a large language model employed by an orchestrator. The data format may include some or all of the different types or modalities described herein (e.g., multimodal, textual, encoded, linguistic, statistical, audio, visual, audiovisual, etc.). In some examples, one or more agents may receive data and aggregate the content of the data in a natural language format for provision to a large language model.

[0238] Transmitting the natural language summary of the data record and the additional data record in response to the input may include transmitting the natural language summary of the data record and the additional data record to a computing device for display at the computing device. The transmitted natural language summary of the data record and the additional data record may depend on the technical functionality of one or more large language models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more large language models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more large language models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more large language models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more large language models and / or the one or more machine learning models may include emulation of the one or more large language models and / or the one or more machine learning models using software.

[0239] The orchestrator may include one or more large language models. The agents may include one or more large language models, and the additional agents may include one or more additional large language models. A first agent (of the one or more agents) may use one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve one or more data records from an unstructured dataset. A second agent (of the one or more agents) may use one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more additional data records from a structured dataset. The first agent may instruct the one or more unstructured data retrieval tools based on embeddings in a vector store. The second agent may instruct the one or more structured data retrieval tools based on a data model that describes one or more types of relationships in the data model. Selecting a first agent from a plurality of different agents based on the processed input using the orchestrator may also include generating an intermediate input based on a first portion of the input and routing the second input to the agent. Selecting, using the orchestrator, a second agent from the plurality of different agents based on the processed input may also include generating a third input based on the second portion of the input and routing the third input to the additional agent. The first agent and the second agent may execute in parallel and perform their respective searches in parallel.

[0240] According to an example disclosed herein, a system is provided that includes: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the system to: select, using an orchestrator, two or more agents from a plurality of different agents based on a received prompt; retrieve, using the two or more agents, one or more data records from an unstructured dataset and additional data records from a structured dataset; generate, using the orchestrator, a natural language summary of the one or more data records from the unstructured dataset and the additional data records from the structured dataset; and transmit, in response to the prompt, the natural language summary of the data records. The selected two or more agents from the plurality of different agents may include a first agent configured to retrieve data records from the unstructured dataset and a second agent configured to retrieve additional data records from the structured dataset. The data records may be retrieved from the unstructured dataset by the first agent, and the additional data records may be retrieved from the structured dataset by the second agent.

[0241] The first agent may instruct one or more unstructured data retrieval tools to retrieve one or more data records from a structured data set. The first agent may be configured to generate instructions for the one or more unstructured data retrieval tools. The first agent may generate instructions in a format and / or language that the one or more unstructured data retrieval tools are configured to receive. The second agent may instruct one or more structured data retrieval tools to retrieve one or more additional data records from a structured data set. The second agent may be configured to generate instructions for the one or more structured data retrieval tools. The second agent may generate instructions in a format and / or language that the one or more structured data retrieval tools are configured to receive. The first agent and / or the second agent may employ (e.g., through training) one or more machine learning models configured to generate instructions for the one or more data retrieval tools. The one or more machine learning models employed by the one or more agents may be configured to determine instructions for the tool based on the format and / or language of the instructions that the tool is configured to receive through training. That is, the machine learning model can be configured through training to output instructions in a given format and / or language for a given tool, and / or can be configured through training to output instructions in different formats and / or languages ​​for different tools.

[0242] The system may include a code generation agent and / or tool. The code generation agent and / or tool may be configured to generate source code, machine code and / or other computer code. In some examples, the code generation agent may be configured to instruct a code generation tool to generate source code, machine code and / or other computer code. The code generation agent may be configured to determine what code is needed and / or what agent, tool and / or other entity or application should generate code for. The code generation agent may instruct the code generation tool to generate code suitable for the function to be received and / or the language and / or format of the receiving agent, tool and / or other entity or application. The code generation agent and / or tool may output code in a format and / or language that is configured to be received by a specific agent, tool and / or other entity or application. Two or more agents may be configured to receive data and convert the data into a natural language format to provide to a large language model. For example, one or more agents may receive data and convert the data into a natural language format to provide to a large language model adopted by an composer. In some examples, one or more agents may receive data and summarize the content of the data in natural language format to provide to a large language model.

[0243] Transmitting the natural language summary of the data record in response to the prompt may include transmitting the natural language summary of the data record to a computing device for display at the computing device. The transmitted natural language summary of the data record may depend on the technical functionality of one or more large language models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more large language models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more large language models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more large language models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more large language models and / or the one or more machine learning models may include emulation of the one or more large language models and / or the one or more machine learning models using software.

[0244] The orchestrator may include one or more large language models. The first agent may employ one or more large language models, and the additional agent may employ a different large language model. The first agent may use one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve one or more first data records from an unstructured dataset. The second agent may use one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from a structured dataset. The first agent may instruct the one or more unstructured data retrieval tools based on embeddings in a vector store. The second agent may instruct the one or more structured data retrieval tools based on a data model that describes one or more types of relationships of the data model.

[0245] Selecting, using the orchestrator, a first agent from among the plurality of different agents based on the processed input may further include generating a second input based on the first portion of the input and routing the second input to the first agent. Selecting, using the orchestrator, a second agent from among the plurality of different agents based on the processed input may further include generating a third input based on the second portion of the input and routing the third input to the second agent. The first agent and the second agent may execute in parallel and perform their respective searches in parallel.

[0246] According to an example disclosed herein, a non-transitory computer-readable medium is provided, comprising instructions that, when executed, cause one or more processors to: process an input using an orchestrator; select a first agent from a plurality of different agents based on the processed input using the orchestrator; retrieve one or more first data records from an unstructured data set using the first agent; select a second agent from a plurality of different agents based on the processed input using the orchestrator; retrieve one or more second data records from a structured data set using the second agent; generate a natural language summary of the one or more first data records and one or more second data records using the orchestrator; and transmit the natural language summary of the one or more first data records and one or more second data records to a recipient associated with the input.

[0247] The first agent may instruct one or more unstructured data retrieval tools to retrieve one or more data records from a structured data set. The first agent may be configured to generate instructions for the one or more unstructured data retrieval tools. The first agent may generate instructions in a format and / or language that the one or more unstructured data retrieval tools are configured to receive. The second agent may instruct one or more structured data retrieval tools to retrieve one or more additional data records from a structured data set. The second agent may be configured to generate instructions for the one or more structured data retrieval tools. The second agent may generate instructions in a format and / or language that the one or more structured data retrieval tools are configured to receive. The first agent and / or the second agent may employ (e.g., through training) one or more machine learning models configured to generate instructions for the one or more data retrieval tools. The one or more machine learning models employed by the one or more agents may be configured to determine instructions for the tool based on the format and / or language of the instructions that the tool is configured to receive through training. That is, the machine learning model can be configured through training to output instructions in a given format and / or language for a given tool, and / or can be configured through training to output instructions in different formats and / or languages ​​for different tools.

[0248] Code generation agents and / or tools can be used to generate source code, machine code and / or other computer code. In some examples, the code generation agent can be configured to instruct the code generation tool to generate source code, machine code and / or other computer code. The code generation agent can be configured to determine what code is needed and / or for which agent, tool and / or other entity or application to generate code. The code generation agent can instruct the code generation tool to generate code suitable for the function to be received and / or the language and / or format of the receiving agent, tool and / or other entity or application. The code generation agent and / or tool can be configured to output code in the format and / or language received by a specific agent, tool and / or other entity or application. For example, two or more agents can be configured to receive data and convert the data into a natural language format to provide to a large language model. For example, one or more agents can receive data and convert the data into a natural language format to provide to a large language model adopted by an assembler. In some examples, one or more agents can receive data and summarize the content of the data in a natural language format to provide to a large language model.

[0249] Transmitting a multimodal summary (e.g., a natural language summary) of the data record in response to a prompt may include transmitting the natural language summary of the data record to a computing device for display at the computing device. The transmitted natural language summary of the data record may depend on the technical functionality of one or more multimodal models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more large language models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more large language models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more large language models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more large language models and / or the one or more machine learning models may include emulation of the one or more large language models and / or the one or more machine learning models using software.

Claims

1. A method comprising: Managing, using an orchestrator, a plurality of agents to generate responses to inputs, wherein the orchestrator employs one or more multimodal models to process or deconstruct a prompt into a series of instructions for different agents, wherein each agent employs one or more machine learning models to process different inputs or different portions of inputs associated with the prompt; Instructing, using the orchestrator, to one or more of the plurality of agents a search request related to the input; receiving data from a plurality of data domains from one or more of the plurality of agents based on instructions from the orchestrator; analyzing, with the orchestrator, the received data to formulate one or more responses to the prompt, wherein the orchestrator provides additional retrieval requests to the one or more agents to retrieve additional data that satisfies contextual validation criteria associated with the input; as well as At least one of the following is outputted, using the orchestrator: a validated response from one or more responses to an input that satisfies contextual validation criteria; and a portion of data retrieved by the one or more agents that is relevant to the input.

2. The method according to claim 1, wherein The orchestrator generates intermediate instructions to the plurality of agents, wherein the intermediate instructions include at least one of: a portion of the input; questions related to the input generated by the one or more multimodal models; and follow-up questions related to answers generated by the one or more multimodal models.

3. The method according to claim 1, wherein Outputting the portion of the data retrieved by the one or more agents that is relevant to the input includes a source reference for at least a portion of the validated response.

4. The method according to claim 1, wherein The contextual validation criteria include a threshold for identifying source material from an enterprise data system for validating the response.

5. The method according to claim 1, wherein Managing the plurality of agents includes iterative processing or a plurality of instructions from the orchestrator.

6. The method according to claim 1, wherein Retrieve data from multiple data domains including time series data, structured data, and unstructured data.

7. The method according to claim 2, wherein: At least one of the one or more agents instantiates a tool to operate on the instructions, retrieved data, or the intermediate instructions.

8. The method according to claim 7, wherein: The operation includes at least one of calculation, translation, formatting, and visualization.

9. The method according to claim 1, wherein: The one or more agents are trained on specific machine learning models in different domains.

10. The method according to claim 1, wherein At least one agent employs a type system to unify incompatible data from different data sources.

11. A method comprising: selecting, using an orchestrator, one or more agents from a plurality of different agents based on pre-processed input generated by the orchestrator from the initial input; Retrieving, using the selected one or more agents, data records from the unstructured dataset and additional data records from the structured dataset; generating, using the orchestrator, a natural language summary of the data records from the unstructured dataset and the additional data records from the structured dataset; as well as A natural language summary of the one or more data records and the one or more additional data records is transmitted as a response to the input.

12. The method according to claim 1, wherein The orchestrator includes one or more multimodal models including at least a large language model.

13. The method according to claim 11, wherein The agent includes one or more multimodal models, and additional agents in the plurality of different agents include one or more additional multimodal models.

14. The method according to claim 13, wherein The agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve the one or more data records from the unstructured dataset.

15. The method according to claim 13, wherein The additional agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve the one or more additional data records from the structured data set.

Images