Generative artificial intelligence crawling and partitioning

Abstract

A plurality of different data fields of an enterprise information environment are scanned. A plurality of data records of a plurality of enterprise data sources of different data domains are partitioned. The partitioning generates one or more respective data record segments for each of the plurality of data records. Respective contextual metadata is generated for each of the one or more respective data record segments. Each respective contextual metadata indicates a semantic or contextual description of a respective data record segment, and at least one of the respective contextual metadata is capable of facilitating determining a relationship between one of the respective data record segments of a particular respective data record and another of the respective data segments of another respective data record. A respective segment embedding is generated for each data record segment based on the respective contextual metadata, and the segment embedding is stored in an embedded data store.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence and machine learning. More specifically, the present disclosure relates to intelligent crawling and chunking for generative artificial intelligence. Background Art

[0002] Artificial intelligence (AI) is a branch of computer science used to develop software that enables computer systems to perform tasks that mimic human cognitive intelligence, such as visual perception, speech recognition, decision making, and language translation. Traditional methods for storing and retrieving information typically involve databases and applications for indexing and locating specific files for searches. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 A diagram depicts an example intelligent crawling and chunking subsystem that processes data records of a multi-domain computing environment for efficient storage and retrieval operations, according to some embodiments.

[0004] Figure 2 A diagram depicting an example layered architecture and environment for an enterprise generative artificial intelligence system, according to some embodiments.

[0005] Figure 3 A diagram depicting an example logical flow of an enterprise generative artificial intelligence system, according to some embodiments.

[0006] Figure 4 A diagram depicts an example network system for enterprise generative artificial intelligence, according to some embodiments.

[0007] Figure 5 A diagram depicts an example enterprise generative artificial intelligence system, in accordance with some embodiments.

[0008] Figure 6 Flowchart depicting an example generative AI unstructured data and structured data retrieval process.

[0009] Figure 7 A diagram depicting an example logical flow of an enterprise generative artificial intelligence system, according to some embodiments.

[0010] Figure 8A Depicted is a flow diagram of an example iterative generative artificial intelligence process using unstructured data, according to some embodiments.

[0011] Figures 8B to 8C Depicted is a flow diagram of an example non-iterative generative artificial intelligence process using unstructured data, in accordance with some embodiments.

[0012] Figure 9 Depicted is a flow diagram of an example iterative generative artificial intelligence process using unstructured data, according to some embodiments.

[0013] Figure 10 Depicted is a flow diagram of an example generative artificial intelligence process using unstructured and structured data, according to some embodiments.

[0014] Figure 11 Depicted is a flow diagram of an example generative artificial intelligence process using unstructured and structured data, according to some embodiments.

[0015] Figure 12 A flow diagram depicts an example of a non-iterative generative artificial intelligence process using unstructured data, according to some embodiments.

[0016] Figure 13 A flow diagram depicts an example of a generative artificial intelligence process using structured data, according to some embodiments.

[0017] Figure 14 A flowchart depicts example operation of an enterprise generative artificial intelligence system according to some embodiments.

[0018] Figure 15 A flow diagram depicts an example method of intelligent crawling and chunking according to some embodiments.

[0019] Figure 16 A diagram depicts an example data record pre-processing and information retrieval process in accordance with some embodiments.

[0020] Figure 17 is a diagram of an example computer system for implementing features disclosed herein, according to some embodiments. DETAILED DESCRIPTION

[0021] Generative AI uses machine learning algorithms to mimic human cognitive intelligence and generate content. This content can take the form of text, audio, video, images, and more. Content in enterprise computing environments is typically distributed across disparate, potentially incompatible, siloed, and access-controlled data sources. Supporting efficient search capabilities is further complicated by the need for subject matter expertise or context-specific knowledge.

[0022] Disclosed herein is an enterprise generative artificial intelligence architecture that can intelligently and efficiently crawl and index diverse data records (e.g., data records of one or more enterprise systems) across various domains using contextual information (e.g., contextual metadata) to provide improved data record identification and retrieval, access control (e.g., role-based access), and mapping relationships between data records. In one example, contextual information can prevent some users from accessing (e.g., viewing, retrieving) certain data records and improve similarity assessments used in retrieval operations (e.g., of a generative artificial intelligence process). As a result, the system described herein can provide more accurate and reliable results that are also faster and more secure than prior art techniques.

[0023] In some embodiments, the enterprise generative artificial intelligence system can crawl, chunk, and index a corpus of data records. Data records can include documents (e.g., PDF, text, html, markdown source code or other source code), database tables, information generated by applications (e.g., artificial intelligence application insights), images, audio-visual files, executable files, and / or models (e.g., data models, machine learning models, large language models, multimodal models), etc. More specifically, the enterprise generative artificial intelligence system pre-processes and chunks data records across different domains of the enterprise (e.g., data domains, industry-specific domains). The chunking process partitions the data records into segments (or chunks) and can insert and / or append contextual information for the segments (e.g., as headers for the segments).

[0024] Context information can include, for example, one or more attributes describing a segment (e.g., the type of data record, the size of the segment, a semantic or contextual description of the segment, access control restrictions or permissions, etc.). A segment can include a paragraph of a text document, a portion of a database table, a sub-model of a model, etc. Segmentation and / or context information can be stored for efficient retrieval (e.g., as part of a generative artificial intelligence process). For example, segmentation and / or context information can be stored as embeddings (e.g., vector embeddings) that can allow efficient retrieval. In one example, context information can include explicit and / or inferred references between segments and / or data records. For example, a reference can indicate a relationship that can be used (e.g., traversed) when performing similarity evaluation or other aspects of a retrieval operation.

[0025] Context information can include context metadata access control to enhance enterprise security. In some implementations, context information provides user-based and / or role-based access control. For example, context information can indicate user roles that can access corresponding segments and / or data records and / or user roles that cannot access corresponding segments and / or data records. Context information can be represented in an embedding so that retrieval operations are prevented from accessing and / or identifying specific data records (e.g., sensitive data records). Context information can be included and / or represented in context metadata and / or context metadata can be generated from context information.

[0026] This paper discloses an architecture for enterprise generative AI to transform the interaction with enterprise information that fundamentally changes the human-computer interaction (HCI) model for enterprise software. Enterprises running sensitive workloads in cloud-native, on-premises, or air-gapped environments can implement an enterprise generative AI architecture to generate enterprise-wide insights in response to simple intuitive inputs, using intelligent agents that develop and orchestrate complex operations, using tools for rapid location and retrieval. The enterprise generative AI architecture enables enterprise users to ask open-ended, multi-level, context-specific questions that are processed using generative AI with machine learning to understand the request, identify relevant information, and generate new context-specific insights using predictive analytics. The enterprise generative AI architecture supports simplified human-computer interaction with an intuitive natural language interface and advanced accessibility features for adaptable forms of input (including but not limited to text, audio, video, images, etc.).

[0027] Conventional generative artificial intelligence processes are computationally inefficient, often contain erroneous or biased information, cannot effectively handle different types of inputs and outputs, cannot effectively utilize different data sources with different data formats, and cannot effectively interact with other machine learning systems or effectively utilize information across different domains. These problems and those discussed above are addressed by the enterprise generative artificial intelligence systems and processes discussed in this article. More specifically, enterprise generative artificial intelligence systems can efficiently provide more accurate and reliable results than conventional generative artificial intelligence solutions, while consuming less computing resources and requiring shorter processing time. In addition, enterprise generative artificial intelligence systems can adopt various models that effectively provide cross-domain functionality. Enterprise generative artificial intelligence systems can also use a combination of agents and tools to efficiently process various inputs received from different data sources (e.g., with different data formats) and return results in a common data format (e.g., natural language).

[0028] The enterprise generative AI architecture includes an orchestrator agent (or simply orchestrator) that oversees, controls, and / or otherwise manages a number of different agents and tools. The orchestrator may include one or more machine learning models and may perform supervisory functions, such as routing inputs (e.g., queries, instruction sets, natural language input or other human-readable input, machine-readable input) to specific agents to complete a specified set of tasks (e.g., retrieval requests specified by the orchestrator to answer queries). The machine learning models may include some or all of the different types or modalities of models described herein (e.g., multimodal machine learning models, large language models, data models, statistical models, audio models, vision models, audio-visual models, etc.). Agents may include one or more multimodal models (e.g., large language models) to use a variety of different tools to complete specified tasks. Different agents may use various tools to execute and process unstructured data retrieval requests, structured data retrieval requests, and API calls (e.g., for accessing AI application insights). A tool may include one or more specific functions and / or machine learning models to complete a given task (or set of tasks).

[0029] The agent can be adapted to perform in different ways based on the context. The context can relate to a specific domain (e.g., an industry), and the agent can employ specific models (e.g., large language models, other machine learning models, and / or data models) that have been trained on industry-specific datasets (such as healthcare datasets, etc.). A specific agent can use a healthcare model when receiving input associated with a healthcare environment, and can also be easily and efficiently adapted to use different models based on different inputs or contexts. In fact, some or all of the models described herein can be trained for specific domains in addition to or instead of more general purposes. The enterprise generative AI architecture leverages domain-specific models to produce accurate context-specific retrieval and insights.

[0030] The orchestrator manages agents to efficiently process different inputs or different parts of inputs. For example, the inputs may require the system to access and retrieve data records from different data sources (e.g., unstructured data stores, structured data stores, and time series data stores), database tables from different types of databases, and machine learning insights from different machine learning applications. Different agents can handle each of these requests independently and in parallel, greatly improving computational efficiency.

[0031] Agents can process different data returned by different agents and / or tools. For example, large language models typically receive input in natural language format. Agents can receive information in non-natural language format from tools (e.g., database tables, images, audio) and transform it into natural language that describes the tool's output in a format understood by the large language model. The large language model can then process this input to "answer" or otherwise satisfy the initial input.

[0032] Figure 1 A diagram 100 depicts an example intelligent crawling and chunking subsystem that processes data records of a multi-domain computing environment for efficient storage and retrieval operations, according to some embodiments. Figure 1 In the example of FIG, a multi-domain computing environment includes enterprise information sources 102, external information sources 103, and an intelligent crawling and segmentation subsystem 120. Enterprise information sources 102 include information sources across different enterprise domains 104. Enterprise domains 104 can include the enterprise's data domains (e.g., documents, tabular data, insights derived from artificial intelligence applications, web content, or other data sources). Enterprise domains 104 can also include the enterprise's industry-specific domains (e.g., healthcare domains, defense domains, etc.). External information sources 103 include external domains 108 outside the enterprise, such as news sources and weather sources.

[0033] The intelligent crawling and chunking subsystem 120 can function to crawl, chunk, pre-process, parse, and / or otherwise process enterprise data records 105-106 across various enterprise domains 104. The intelligent crawling and chunking subsystem 120 can further crawl, chunk, pre-process, parse, and / or otherwise process external data records 109-110. For example, a type system can be used to ingest, transform, and / or package external data records 109-110 (e.g., without changing the original content) to be compatible with the enterprise information sources 102 and / or systems described herein (e.g., the intelligent crawling and chunking subsystem 120 and / or the enterprise generative artificial intelligence system 402).

[0034] More specifically, the crawling module 122 can function to scan and / or crawl different data sources (e.g., enterprise information sources 102, external information sources 103) across different domains 104 and 108. This can identify existing data records 105-106 and 109-110, new data records 105-106 and 109-110, and / or updated data records 105-106 and 109-110. The crawling module 122 can notify the chunking module 124 of the new data records 105-106 and 109-110 and the updated data records 105-106 and 109-110, and the chunking module 124 can chunk the data records 105-106 and 109-110. In some embodiments, the crawling module 122 can function and / or trigger operations periodically, on demand, and / or in real time.

[0035] In some implementations, information sources 102 and / or 106 may include a model registry that stores various models (e.g., machine learning models, large language models, multimodal models). Models can be trained on general datasets and / or domain-specific datasets. The processes described herein can be applied to various model registries. For example, a model can be associated with an embedding value (e.g., generated by an embedding model) to facilitate model retrieval.

[0036] Chunking module 124 can function to process (e.g., chunk) a corpus of data records (e.g., of one or more enterprise systems and / or external systems) for processing by various systems (e.g., enterprise generative artificial intelligence system 402). Chunking module 124 can partition the data records and insert or append a corresponding header for each chunk. The header can include one or more attributes describing the chunk. A segment can include a header and a paragraph of a text document, a portion of a database table, a model or sub-model, etc. For simplicity, a reference to a paragraph can include the segment and / or other content of the segment (e.g., text). The segment can be stored in a segment data store (e.g., segment data store 126). Chunking can be rule-based.

[0037] In some implementations, the chunking module 124 may pre-process the data records and / or segments to generate corresponding context information. In some embodiments, the context information may be included and / or represented in context metadata, and / or the context metadata may be generated from the context information. The context information may improve security and the accuracy and reliability of associated retrieval operations. In one example, the context information includes context metadata. The context information may include references between the segments and / or data records 105-106 and 109-110. For example, the references may indicate relationships that can be used (e.g., traversed) when performing similarity assessments or other aspects of a retrieval operation (e.g., by one or more of the agents 506). The context information may also include information that can assist large language models in generating plans and / or answers. For example, the chunking module 124 may generate context information for a structured data chunk (or paragraph) that includes a natural language description of the data records 105-106 and 109-110 and the locations of the related data records 105-106 and 109-110.

[0038] Context information can include access control. In some implementations, context information provides user-based access control (e.g., role-based access control) to the associated data records 105-106 and 109-110 and / or segmentation. More specifically, context information can indicate the user role that can access the corresponding segmentation and / or data record and / or the user role that can not access the corresponding segmentation and / or data record. Context information can be stored in the header of the data records 105-106 and 109-110 and / or data record segmentation. Context information can maintain references between data records 105-106 and 109-110 and / or data record segmentation. The segmentation module 124 can generate context information before, after, or simultaneously with generating the associated embedding. For example, context information can be used to create an embedding, or context information can be utilized to enhance the embedding. The contextual information can be used by the chunking module 124 to map relationships between data records 105-106 and 109-110 and / or segments of one or more enterprises or enterprise systems and store these relationships in a data model. In one example, the chunking module 124 implements the word2vec algorithm. In some implementations, the chunking module 124 utilizes a model trained on a domain-specific (or industry-specific) dataset.

[0039] In some embodiments, the chunking module 124 can perform some or all of the functionality described herein periodically (e.g., in batches), on-demand, and / or in real-time. For example, the chunking module 124 can trigger the chunking described herein periodically, on-demand, manually, and / or automatically. In some implementations, subsequent chunking operations can incorporate only changes (e.g., "deltas") relative to previous chunking operations.

[0040] In some embodiments, the embedding generator module 128 can generate embeddings using a variety of different embedding models (e.g., single vector embedding models, token vector embedding models, etc.). Embeddings can be generated from data records or segments. Embeddings can also be generated from data records and contextual information. Thus, for example, retrieval operations can more accurately perform similarity analysis.

[0041] In some embodiments, the embedding generator module 128 can generate segment embeddings based on the corresponding segments and the corresponding contextual metadata. For example, the segment embeddings can include vector embeddings that can be used as part of a similarity machine learning process for determining similarities between data records 105-106 and 109-110 and / or segments. Thus, similarity can be readily determined based on various segments and corresponding contextual information (e.g., as part of a generative artificial intelligence retrieval operation).

[0042] Embeddings can be generated based on both structured and unstructured data records and / or segments. The embedding generator module 128 can include one or more models (e.g., embedding models, deep learning models) that can convert and / or transform the data records 105-106 and 109-110 into vector representations, where vectors for semantically similar segments and / or data records (e.g., the content of the data records) are close together in the vector space.

[0043] In some embodiments, the embedding generator module 128 can generate embeddings using one or more embedding models (e.g., an implementation of the ColBERT embedding model). Embeddings can include digital representations of unstructured and / or structured data records and / or segments that capture the semantic or contextual meaning of the data records and / or segments. For example, an embedding can be represented by one or more vectors. The embeddings can be used when retrieving data records or segments (e.g., from the embedding data store 130) and performing similarity evaluation or other aspects of the retrieval operation. The embeddings can be stored in an embedding index (e.g., the vector data store 130). In some embodiments, the embedding data store is a vector store, which is a type of database specifically optimized for storing and retrieving embeddings using similarity heuristics (e.g., approximate nearest neighbor (ANN) algorithms) that can be implemented by various agents and / or tools.

[0044] Figure 2 Diagram 200 depicts an example layered architecture and environment for an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. Figure 2 In the example of FIG, the enterprise generative artificial intelligence system architecture and environment includes a hierarchy of layers. More specifically, the hierarchy of layers includes an input layer 202, a supervision layer 210, an agent layer 220, an agent and tool layer 230, a tool and data model layer 250, and an external layer 280. It will be understood that these layers are shown by way of example, and other examples may include any number of such layers (e.g., any number of layers 220 and 230).

[0045] Input layer 202 represents the layer of the enterprise generative artificial intelligence system architecture that receives input (e.g., queries, complex inputs, and / or instruction sets, etc.) from a user or system. For example, an interface module of the enterprise generative artificial intelligence system can receive input.

[0046] The supervisory layer 210 represents the following layer of the enterprise generative artificial intelligence system architecture, which includes one or more large language models (e.g., of an orchestrator module) that can develop plans for responding to input received in the input layer 202. The plans can include a set of specified tasks (e.g., retrieval tasks and API call tasks, etc.). In one example, the supervisory layer 210 can provide the pre-processing and post-processing functionality described herein, as well as the functionality of the orchestrator and understanding modules described herein. The supervisory layer 210 can coordinate with one or more of the subsequent layers 220-280 to perform the specified set of tasks.

[0047] The agent layer 220 represents the layer of the enterprise generative artificial intelligence system architecture that includes agents that can perform a specified set of tasks. Figure 2In the example of , the agent layer 220 includes a machine learning insight agent 222, an information retrieval agent 224, a dashboard agent 226, and an optimizer agent 228. Each of the agents 224-228 can include a large language model that provides reasoning functionality for completing its assigned portion of a specified set of tasks. More specifically, the agents 224-228 can instruct agents and tools in any number of subsequent layers (e.g., layer 230) that may exist to perform tasks. For example, the machine learning insight agent 222 can instruct the text processing tool 232 to perform a text processing task (e.g., convert the output of an artificial intelligence application into natural language), instruct the image processing tool 234 to perform an image processing task (e.g., generate a natural language summary of an image output from an artificial intelligence application), instruct the time series tool 236 to obtain summarized time series data (e.g., time series data output from an artificial intelligence application), and instruct the API tool 238 to perform an API call task (e.g., execute an API call to trigger or access an artificial intelligence application).

[0048] The information retrieval agent 224 can collaborate and / or coordinate with several different agents to perform retrieval tasks. For example, the information retrieval agent 224 can instruct the unstructured data retriever agent 240 to receive unstructured data records, instruct the structured data retriever agent 242 to retrieve structured data records, and instruct the type system retriever agent 244 to obtain one or more data models (or subsets of data models) and / or types from the type system. The type system provides compatibility across different data formats, protocols, operating languages, different systems, etc. A type can encapsulate some or all of the different types or modalities described herein (e.g., multimodal, textual, encoded, linguistic, statistical, audio, visual, audiovisual, etc.).

[0049] Specifically, type systems address data operability across various programming languages, inconsistent data structures, and incompatible software application programming interfaces. Type systems provide a data abstraction that defines an extensible type model that enables the dynamic addition of new properties, relations, and functions without expensive development cycles. Type systems can be used as domain-specific languages ​​(DSLs) within the platform that developers, applications, or UIs use to access data. Type systems provide the ability to interact with data to process, predict, or parse it based on one or more type or function definitions within the type system.

[0050] A type definition can be a canonical type declared in metadata using a syntax similar to that used by types persisted in relational or NoSQL data stores. The canonical model in the type system is an application-independent model, enabling all applications to communicate with each other in a common format. Unlike standard types, a canonical type consists of two parts: a canonical type definition and one or more transformation types. The canonical type definition defines the interface for integration, and the transformation type is responsible for transforming the canonical type into the corresponding type. Using the transformation type, the integration layer can transform the canonical type into the appropriate type.

[0051] For example, a data model may include various different types (e.g., in a tree or graph structure), and each type may describe data fields, operations, functions, and the like. Each type may represent a different object (e.g., a real-world object, such as a machine or sensor in a factory) or system (e.g., a computing cluster, an enterprise data store, a file system), and each type may include a large language model context that provides context for a large language model to design or update a plan. For example, the context may include a natural language summary or description of the type (e.g., a description of the object represented, its relationships to other types or objects, and associated methods and functions, etc.). Types may be defined in a natural language format for efficient processing by a large language model. The type system retriever agent 244 may traverse the data model 254 to retrieve subsets of the data model 254 and / or types of the data model 254. The structured data retriever agent 242 may then use this retrieved information to efficiently retrieve structured data from a structured data source (e.g., a structured data source structured or modeled according to the data model 254).

[0052] The dashboard agent 226 may be configured to generate one or more visualizations and / or graphical user interfaces, such as dashboards, etc. For example, the dashboard agent 226 may execute tools 252-5 and 252-6 to generate a dashboard based on information retrieved by other agents and / or information output by other agents (e.g., natural language summaries of associated tool outputs).

[0053] The optimizer agent 228 can be configured to perform various prescriptive analytical functions and mathematical optimizations 252-7 to assist in computing answers to various questions. For example, the large language model 206 can use the optimizer agent 228 to generate a plan, determine a set of prescriptive tasks, and determine whether more information is needed to generate a final result.

[0054] Tools and data model layer 250 is intended to represent a layer of the enterprise generative AI system architecture that includes tools 252 and data models 254. Agents 240-242 can execute tools 252 to retrieve information from various applications and data stores 282 in external layer 280 (e.g., external to the enterprise generative AI system). Tools 252 can include connectors that can connect to systems and data stores external to the enterprise generative AI system.

[0055] Figure 3 A diagram 300 depicts an example logic flow of an enterprise generative artificial intelligence system according to some embodiments. As shown, initial input 302 is received by the system from a user (e.g., natural language input) or another system (e.g., machine-readable input).

[0056] The orchestrator agent (or simply orchestrator) may pre-process the input in step 304. Pre-processing may include, for example, acronym handling, translation handling, punctuation handling, input recognition (e.g., identifying different parts of input 302 to be processed by different agents). The orchestrator may further process input 302 using a multimodal model (e.g., a large language model) to create a plan for determining the outcome of the input (step 312). The plan may include a set of specified tasks, such as a structured data retrieval task, an unstructured data retrieval task, a time series processing task, and a visualization task. In some embodiments, the plan may specify which tools 308 are to be used to perform the tasks, and the orchestrator may select an agent based on the specified tools. In some embodiments, the plan may specify which agents are to be used to perform the tasks, and the agents may independently specify which tools 108 are to be used to perform the tasks.

[0057] continue Figure 3 For example, the orchestrator routes the pre-processed input to an agent 306 for further processing. More specifically, the orchestrator may use one or more multimodal models (e.g., language, video, audio, statistical models, etc.) and / or other machine learning models to interpret the input 302 to select an appropriate agent 306 and an appropriate tool 308. For example, the orchestrator may determine that a first portion of the input requires a database query, while another portion of the input requires an API call. The orchestrator may appropriately route the first portion of the input to an appropriate agent 306-1 (e.g., a structured data retrieval agent) and the second portion of the input to another agent 306-2 (e.g., an API agent). There may be any number of such agents 306 accessing any number of different tools 308. The orchestrator may also instruct the agents 306 to operate in parallel and / or serially.

[0058] Agent 306 can select appropriate tools 308 to complete a specified set of tasks (e.g., tasks specified by the orchestrator). Tools 308 can make appropriate function calls to retrieve different data records and other functions. Data records can include unstructured data records (e.g., documents and text data stored on a file system in formats such as PDF, DOCX, .MD, HTML, TXT, PPTX, image files, audio files, video files, and application output), structured data records (e.g., database tables or other data records stored according to a data model or type system), time series data records (e.g., sensor data, artificial intelligence application insights), and / or other types of data records (e.g., access control lists). Agent 306 can transform the different data records into a common format (e.g., a natural language format) that can be post-processed (step 310) by a large language model (e.g., the same or a different large language model that performed pre-processing). More specifically, post-processing can take the tool output (and / or the transformed tool output) and generate a final result that satisfies the initial input (step 312). For example, the orchestrator can use one or more large language models to determine the result. If the orchestrator determines that there is not enough information to satisfy the initial input, the orchestrator may iteratively repeat some or all of the above steps until a stopping condition is met and / or there is enough information to generate a final result (step 312).

[0059] Figure 4 A diagram 400 depicts an example network system for enterprise generative artificial intelligence, according to some embodiments. Figure 4 In the example, the network system includes enterprise generative artificial intelligence system 402, enterprise systems 404-1 through 404-N (individually enterprise system 404 and collectively enterprise systems 404), external systems 406-1 through 406-N (individually external system 406 and collectively external systems 406), and communication network 408.

[0060] Enterprise generative artificial intelligence system 402 can function to iteratively and non-iteratively generate machine learning model inputs and outputs to determine a final output (e.g., an "answer" or "result") in response to initial input (e.g., provided by a user or another system). In some embodiments, the functionality of enterprise generative artificial intelligence system 402 can be performed by one or more servers (e.g., cloud-based servers) and / or other computing devices. Enterprise generative artificial intelligence system 402 can be implemented using a type system and / or a model-driven architecture.

[0061] In various implementations, the enterprise generative artificial intelligence system 402 can provide a variety of different technical features, such as efficiently processing and generating complex natural language input and output, generating synthetic data (e.g., to supplement customer data obtained during the onboarding process or otherwise fill in data gaps), generating source code (e.g., application development), generating applications (e.g., artificial intelligence applications), providing cross-domain functionality, and countless other technical features not provided by traditional systems. As used herein, synthetic data can refer to content generated on the fly (e.g., by a large language model) as part of the processes described herein. Synthetic data can also include ephemeral content that is not retrieved (e.g., temporary data that does not exist in the database), as well as a combination of retrieved information, queried information, and / or model output.

[0062] The enterprise generative artificial intelligence system 402 can provide and / or enable intuitive, non-complex interfaces to quickly execute complex user requests with improved access, privacy, and security enforcement. The enterprise generative artificial intelligence system 402 can include a human-machine interface for receiving natural language queries and presenting relevant information with predictive analytics from the enterprise information environment in response to these queries. For example, the enterprise generative artificial intelligence system 402 can understand the language, intent, and / or context of the user's natural language query. The enterprise generative artificial intelligence system 402 can execute the user's natural language query to discern relevant information from the enterprise information environment to present to the human-machine interface (e.g., in the form of an "answer").

[0063] The generative AI models (e.g., the orchestrator's large language model) of the enterprise generative AI system 402 can interact with agents (e.g., retrieval agents, retriever agents) to retrieve and process information from various data sources. For example, a data source can store data records and / or segments of data records that can be identified by the enterprise generative AI system 402 based on embedded values ​​(e.g., vector values ​​associated with the data records and / or segments). Data records can include tables, text, images, audio, video, code, and / or application output (e.g., predictive analysis and / or other insights generated by an AI application), etc.

[0064] The enterprise generative artificial intelligence system 402 can generate context-based synthetic output based on information retrieved from one or more retriever models. For example, a retriever model (e.g., a retriever model or a retrieval agent) can provide additional retrieved information to a large language model to generate additional context-based synthetic output until context validation criteria are met. Once the validation criteria are met, the enterprise generative artificial intelligence system 402 can output the additional context-based synthetic output as a result or set of instructions (collectively, "answers").

[0065] The enterprise generative artificial intelligence system 402 provides transformed context-based intelligent generation results. For example, the enterprise generative artificial intelligence system 402 can use a natural language interface to process input from enterprise users to quickly locate, retrieve, and present relevant data across the entire corpus of the enterprise's information systems.

[0066] The enterprise generative AI system 402 can process both machine-readable input (e.g., compiled code, structured data, and / or other types of formats that can be processed by a computer) and human-readable input. The input can also include complex input, such as input that includes "and," "or," and / or input that includes different types of information to satisfy the input (e.g., data records, text documents, database tables, and AI insights). In one example, a complex input might be "How many different engineers has John Doe worked with within his engineering department?" This may require the enterprise generative AI system 402 to identify John Doe in a first iteration, identify John Doe's department in a second iteration, determine the engineers in that department in a third iteration, then determine which of these engineers John Doe has interacted with in a fourth iteration, and finally combine these results, or portions thereof, to generate a final answer to the query. More specifically, the enterprise generative AI system 402 can use portions of the results of each iteration to generate contextual information (or simply context), which can then inform subsequent iterations.

[0067] exist Figure 4In the example of , enterprise generative artificial intelligence system 402 includes intelligent crawling and chunking subsystem 403. Intelligent crawling and chunking subsystem 403 can be the same intelligent crawling and chunking subsystem 120. Enterprise systems 404 can include enterprise applications (e.g., artificial intelligence applications), enterprise data stores, client systems and / or other systems of the enterprise information environment. As used herein, an enterprise information environment can include one or more networks (e.g., local, air-gapped, or other) of enterprise systems (e.g., enterprise applications, enterprise data stores), client systems (e.g., computing systems for accessing enterprise systems). Enterprise systems 404 can include different computing systems, applications and / or data stores, as well as enterprise-specific requirements and / or features. For example, enterprise systems 404 can include access and privacy controls. For example, an organization's private network can include an enterprise information environment that includes various enterprise systems 404. Enterprise systems 404 can include, for example, CRM systems, EAM systems, ERP systems, FP&A systems, HRM systems, and SCADA systems. Enterprise systems 404 can include or utilize artificial intelligence applications, and artificial intelligence applications can utilize enterprise systems and data. Enterprise systems 404 may include data flows and management of different processes (e.g., of one or more organizations) and may provide access to systems and users of an enterprise while preventing access from other systems and / or users. It will be understood that in some embodiments, references to an enterprise information environment may also include enterprise systems, and references to enterprise systems may also include enterprise information environments. In various embodiments, the functionality of enterprise systems 404 may be performed by one or more servers (e.g., cloud-based servers) and / or other computing devices.

[0068] External systems 406 may include applications, data stores, and systems external to the enterprise information environment. In one example, enterprise system 404 may be a portion of an organization's enterprise information environment that is inaccessible to users or systems external to the enterprise information environment and / or the organization. Thus, example external systems 406 may include Internet-based systems external to the enterprise information environment, such as news media systems and / or social media systems. In various embodiments, the functionality of external systems 406 may be performed by one or more servers (e.g., cloud-based servers) and / or other computing devices.

[0069] The communication network 408 can represent one or more computer networks (e.g., LANs, WANs, air-gapped networks, and / or cloud-based networks, etc.) or other transmission media. In some embodiments, the communication network 408 can provide communication between systems, modules, engines, generators, layers, agents, tools, orchestrators, data stores, and / or other components described herein. In some embodiments, the communication network 408 includes one or more computing devices, routers, cables, buses, and / or other network topologies (e.g., meshes, etc.). In some embodiments, the communication network 408 can be wired and / or wireless. In various embodiments, the communication network 408 can include a local area network (LAN), a wide area network (WAN), the Internet, and / or one or more networks that can be public, private, IP-based, non-IP-based, local, air-gapped, and / or the like.

[0070] Figure 5 A diagram 500 depicts an example enterprise generative artificial intelligence system 402, according to some embodiments. Figure 5 In the example of FIG, the enterprise generative artificial intelligence system 402 includes a management module 502, an orchestrator module 504, a retrieval agent module 506-1, an unstructured data retriever agent module 506-2, a structured data retriever agent module 506-3, a type system retriever agent module 506-4, a machine learning insight module 506-5, a time series processing agent 506-6, an API agent module 506-7, a mathematical agent module 506-8, a visualization agent module 506-9, a code generation agent module 506-10, an unstructured data retrieval tool 508-1, a structured data retrieval tool 508-2, a text processing tool module 508-3, an image processing tool module 508-4, a time series processing tool module 508-5, an API tool module 508-6, a visualization tool module 508-7, an optimization agent module 506-11, and a plurality of other modules. Optimizer tool module 508-8, filter tool module 508-9, projection tool module 508-10, grouping tool module 508-11, sorting tool module 508-12, restriction tool module 508-13, code generation tool module 508-14, chunking module 510, embedding generator module 512, crawling module 514, understanding module 516, enterprise access control module 518, artificial intelligence traceability module 520, parallelization module 522, model generation module 524, model deployment module 526, model optimization module 528, interface module 530, communication module 532, (one or more) vector data store 540, (one or more) model registry data store 550, (one or more) feature data store 560 and (one or more) enterprise generative artificial intelligence system data store 570.

[0071] In some embodiments, chunking module 510, embedding generator module 512, crawling module 514, embedding data store 540, and a portion of enterprise generative artificial intelligence system data store 570 (e.g., a segmentation data store) may include an intelligent crawling and chunking subsystem (e.g., intelligent crawling and chunking subsystem 120).

[0072] Management module 502 can function to (e.g., create, read, update, delete, or otherwise access) data associated with enterprise generative artificial intelligence system 402. Management module 502 can store or otherwise manage or store data in any of data stores 540-570 and / or one or more other local and / or remote data stores. It will be understood that the data store can be a single data store local to enterprise generative artificial intelligence system 402 and / or multiple data stores remote from enterprise generative artificial intelligence system 402. In some embodiments, the data stores described herein include one or more local and / or remote data stores. Management module 502 can operate manually (e.g., through user interaction with a GUI) and / or automatically (e.g., triggered by one or more of modules 504-530). As with other modules described herein, some or all of the functionality of management module 502 can be included in and / or collaborate with one or more other modules, systems, and / or data stores.

[0073] The orchestrator module 504 can function to generate and / or execute one or more orchestrator agents (or simply orchestrators). The orchestrator can orchestrate, supervise, and / or otherwise control agents 506. In some implementations, the orchestrator includes one or more large language models. The orchestrator can interpret inputs, select appropriate agents for handling queries and other inputs, and route the interpreted inputs to the selected agents. The orchestrator can also perform various supervisory functions. For example, the orchestrator can implement stop conditions to prevent the understanding module from falling into an infinite loop during the iterative context-based generative artificial intelligence process. The orchestrator can also include one or more other types of models to process (e.g., transform) non-text inputs. In addition to or in place of the large language models used for some or all of the agents and / or modules described herein, other models (e.g., other machine learning models, translation models) can also be included.

[0074] In some embodiments, the orchestrator can process data received from various data sources in different formats (which can be processed using natural language processing (NLP) (e.g., using tokenization, stemming, lemmatization, and normalization, etc.) with vectorized data) and can generate pre-trained transformers that are fine-tuned or retrained on specific data tailored to the associated data domain or data application (e.g., SaaS applications, legacy enterprise applications, artificial intelligence applications). Further processing can include data modeling feature inspection and / or machine learning model simulation to select one or more appropriate analysis channels. Example data objects can include accounts, products, employees, suppliers, opportunities, contracts, locations, digital portals, geo-located manufacturers, supervisory control and data acquisition (SCADA) information, open manufacturing system (OMS) information, inventory, supply chain, bill of materials, transportation services, maintenance logs, and service logs.

[0075] In some embodiments, the orchestrator module 504 can use various components, as needed, to inventory or generate objects (e.g., components, functionality, and / or data, etc.) using rich descriptive metadata to dynamically generate embeddings for developing knowledge across a wide range of data domains (e.g., documents, tabular data, insights derived from artificial intelligence applications, web content, or other data sources). In an example implementation, the orchestrator module 504 can utilize some or all of the components described herein, for example. Thus, for example, the orchestrator module 504 can facilitate storage, transformation, and communication to facilitate processing and embedding data. In some implementations, the orchestrator module can create embeddings for multiple data types across multiple vertical industries and knowledge domains, as well as specific enterprise knowledge. Knowledge can be explicitly modeled and / or learned by the orchestrator module 504, the agent 506, and / or the tool 508. In an example, the orchestrator module 504 (and / or the chunking module 510) generates embeddings that are translated or transformed to be compatible with the understanding module 516.

[0076] In some embodiments, the orchestrator 504 can be configured to operate or interface different data domains with components of the enterprise generative artificial intelligence system 402. In one example, the orchestrator module 504 can embed objects from a specific data domain and across data domains, applications, data models, analytical byproducts, artificial intelligence predictions, and knowledge repositories to provide robust search functionality without requiring specialized programming for each different data domain or data source. For example, the orchestrator module 504 can create multiple embeddings for a single object (e.g., an object can be embedded in a domain-specific or application-specific context). In some embodiments, the chunking module 510, in conjunction with the orchestrator module 504, can orchestrate the data domain to embed objects of the data domain in the enterprise information system and / or environment. In some embodiments, the orchestrator 504 can collaborate with the chunking module 510 to provide the embedding functionality described herein.

[0077] In some embodiments, the orchestrator module 504 can cause the agent 506 to perform data modeling to translate the original source data format into a target embedding (e.g., object and / or type, etc.). The data format can include some or all of the different types or modalities described herein (e.g., multimodal, text, encoded, linguistic, statistical, audio, visual, audiovisual, etc.). In an example implementation, the orchestrator module 504 and / or the enterprise generative artificial intelligence system 402 typically employs a type system of a model-driven architecture to perform data modeling to translate the original source data format into the target type. The enterprise generative artificial intelligence system 402 and the knowledge base of the generative artificial intelligence model can create the ability to integrate or combine insights from different artificial intelligence applications.

[0078] As discussed elsewhere herein, in addition to human-readable input, enterprise generative AI system 402 can also process machine-readable input (e.g., compiled code, structured data, and / or other types of formats that can be processed by a computer). Input can also include complex input, such as input that includes "ands" and "ors," input that includes different types of information (e.g., text documents, database tables, and AI insights) to satisfy the input, etc. Orchestrator 504 can decompose these complex inputs (e.g., by using a large language model) for processing by multiple agents 506 (e.g., in parallel).

[0079] As discussed above, the orchestrator module 504 can function to perform and / or otherwise handle various supervisory functions. In some implementations, the orchestrator module 504 can enforce conditions (e.g., stopping conditions, resource allocation, and / or prioritization, etc.). For example, a stopping condition can indicate a maximum number of iterations (or hops) that can be performed before the iterative process terminates. The stopping conditions and / or other features managed by the orchestrator module 504 can be included in the large language model hints and / or the large language model of the orchestrator and / or the understanding module 516 discussed below. In some embodiments, the stopping condition can ensure that the enterprise generative artificial intelligence system 402 will not fall into an infinite loop. This feature can also allow the enterprise generative artificial intelligence system 402 to have the flexibility to have different numbers of iterations for different inputs (e.g., as opposed to having a fixed number of hops). In another example, the orchestrator module 504 can perform resource allocation based on computing conditions, such as virtualization or load balancing. In some implementations, orchestrator module 504 and / or agent 506 include a model that can convert (or transform) images, database tables, and / or other non-text input into a text format (eg, natural language).

[0080] In some embodiments, the orchestrator module 504 can function to collaborate with the agents 506 (e.g., the retrieval agent module 506-1, the unstructured data retriever agent module 506-2, and the structured data retriever agent module 506-3) to iteratively and non-iteratively process inputs to determine output results or answers, determine context and root causes for informing subsequent iterations, and determine whether the large language model (e.g., of the orchestrator 504 and / or the understanding module 516) requires additional information to determine the answer. For example, the orchestrator module 504 can receive a query and instruct the agent 506-1 to retrieve associated information. The retrieval agent module 506-1 can then select the unstructured data retriever agent module 506-2 and / or the structured data retriever agent module 506-3 based on whether the orchestrator module 504 wants to retrieve structured data records or unstructured data records. The appropriate agent 506 can select the corresponding tool and provide the tool output to the orchestrator module 504 and / or the understanding module 516 to determine the final result.

[0081] Orchestrator 504 can also select and swap models as needed. For example, in addition to before or after runtime, orchestrator 504 can also swap out models (e.g., data models, large language models, machine learning models) of enterprise generative artificial intelligence system 402 at runtime or during runtime. For example, orchestrator 504, agents 506, and understanding module 516 can use a particular set of machine learning models for one domain and other models for a different domain. Orchestrator 504 can select and use the appropriate model for a given domain and / or input.

[0082] In some embodiments, the orchestrator 504 can combine (e.g., concatenate) the outputs / results from the various agents to create a unified output. For example, one or more of the agent modules 506 can obtain / output a document (or (one or more) segments thereof) or related information (e.g., a text summary or translation), and another agent module 506 can obtain / output a database table, etc. The orchestrator 504 can then use one or more machine learning models (e.g., a large language model and / or another machine learning model) to combine the outputs / results into a unified output (e.g., having a common data format, such as natural language, etc.).

[0083] In some implementations, orchestrator 504 pre-processes input (e.g., initial input) before sending it to one or more agents 506 for processing. For example, orchestrator 504 may transform a first portion of the input into a SQL query and send it to the Unstructured Data Retriever Agent Module 506-2 agent, and transform a second portion of the input into an API call and send it to the API Agent Module 506-7, etc. In another example, such transformation functionality may be performed by agents 506 instead of or in addition to orchestrator 504.

[0084] The orchestrator module 504 can function to process, extract, and / or transform different types of data (e.g., text, database tables, images, videos, and / or code, etc.). For example, the orchestrator module 504 can accept a database table as input and transform it into natural language describing the database table, which can then be provided to the understanding module 516, which can then process the transformed input into an "answer" or otherwise satisfy the query. In some embodiments, a large language model can be used to process text, while another model can be used to convert (or transform) images, database tables, and / or other non-text input into a text format (e.g., natural language).

[0085] It will be understood that, in some embodiments, orchestrator module 504 may include some or all of the functionality of comprehension module 516. For example, comprehension module 516 may be a component of orchestrator module 504. Similarly, in some embodiments, comprehension module 516 may include some or all of the functionality of orchestrator module 504.

[0086] exist Figure 5In the example of , agent modules 506 include various different example agent modules 506-1 to 506-N. It will be understood that these are shown as examples and that various embodiments may include different agents instead of or in addition to agents 506-1 to 506-N. In some embodiments, each agent 506 includes hardware and / or software and includes one or more large language models, one or more other machine learning models and / or functions to provide reasoning functionality to complete a specified set of tasks. It will be understood that references to agent modules can refer to the agents themselves and / or components that generate and / or execute the agents. In some embodiments, an orchestrator is a type of agent and can be referred to as an orchestrator agent. Therefore, references to an orchestrator can refer to the orchestrator itself and / or components that generate and / or execute the orchestrator.

[0087] In various embodiments, some or all of the agents 506 can process data of different data types and / or data formats. For example, the agent module 506 can receive a database table or an image as input (e.g., received from the tool 508) and translate the table or image into natural language describing the table or image, which can then be output for processing by other modules, models, and / or systems (e.g., the composer module 504 and / or the understanding module 516). In one example, a large language model can be used to process text, while another model can be used to convert (or transform) images, database tables, and / or other non-text inputs into a text format (e.g., natural language).

[0088] The retrieval agent module 506-1 can function to retrieve structured and unstructured data records. In some embodiments, the retrieval agent module 506-1 can coordinate / instruct the unstructured data retriever agent module 506-2 to retrieve unstructured data records, and coordinate / instruct the structured data retriever agent module 506-3 and the type system retriever agent 506-4 to retrieve structured data records. For example, the retrieval agent module 506-1 can collaborate with other agents 506 and tools 508 to generate SQL queries to query an SQL database.

[0089] The unstructured data retriever agent module 506-2 can function to retrieve unstructured data records and / or paragraphs (or segments) of these data records (e.g., from an unstructured data store). Unstructured data records can include, for example, text data stored on a file system in formats such as PDF, DOCX, MD, HTML, TXT, and PPTX.

[0090] In some embodiments, agent 506-2 can use embeddings (e.g., vectors stored in vector storage 540) when retrieving information. For example, agent 506-2 can use similarity evaluation or search on vector data storage 540 to find related data records based on k-nearest neighbors, where embeddings that are closer to each other are more likely to be related.

[0091] In some embodiments, the unstructured data retriever agent module 506-2 implements a read-extract-answer (REA) data retrieval process and / or a read-answer (RA) data retrieval process. More specifically, REA and RA may be appropriate when the system 402 needs to process large amounts of data. For example, a query may identify many different data records and / or paragraphs (e.g., hundreds or thousands of data records and paragraphs). For simplicity, references to data records may include data records and / or paragraphs.

[0092] More specifically, the unstructured data retriever agent module 506-2 can determine whether each data record is relevant to answering the query and filter out irrelevant data records. For example, the agent 506-2 can calculate and assign a relevance score for each retrieved data record (e.g., using a machine learning relevance model). The relevance score can be relative to other retrieved data records. For example, the least relevant data record can be assigned a minimum value (e.g., 0), and the most relevant data record can be assigned a maximum value (e.g., 100). The unstructured data retriever agent module 506-2 can filter out relevant documents (or irrelevant documents). For example, the unstructured data retriever agent module 506-2 can filter out data records with a relevance score below a configurable threshold (e.g., 50). In some embodiments, the number of data records that the unstructured data retriever agent module 506-2 can retrieve for a particular input or query can be user- or system-defined and can also be configurable. For example, the system can define that a maximum of 50 data records can be returned.

[0093] In some embodiments, a large language model (e.g., of the unstructured data retriever agent module 506-2) can identify key points of relevant documents and paragraphs, and then provide the key points to the large language model (e.g., the large language model of the arranger 504). The large language model can provide summaries that can be used to generate query answers (e.g., the summaries can be query answers). This can, for example, allow the system 402 to view a wide variety of concepts and documents (e.g., as opposed to an iterative process). In some embodiments, if the number of documents or paragraphs is below a threshold, the unstructured data retriever agent module 506-2 can skip the "extraction" step (e.g., summarizing key points) and provide the paragraphs directly to the large language model. This can be referred to as an RA process.

[0094] The structured data retriever agent module 506-3 can function to retrieve structured data records and / or their segments (or fragments) from various structured data stores. For example, a structured data record can include tabular data persisted in a relational database, a key-value store, or an external database and modeled or accessed using entity types (or simply types). A structured data record can include a data record structured according to one or more data models (e.g., a complex data model) and / or a data record that can be retrieved based on one or more data models. A structured data record can include a data record stored in a structured data store (e.g., a data store structured according to one or more data models).

[0095] In certain implementations, the data model may include a graph structure of objects or types, and agents 506 and / or tools 508 may traverse the graph in different paths to identify relevant types of the data model (e.g., depending on the query and the plan provided by orchestrator module 504 to answer the query), and may combine multiple tables using complex joins (e.g., as opposed to simply passing a single piece of data from the single table and operating on the single table). The paths may be stored in a data store (e.g., vector data store 540) for efficient retrieval.

[0096] In some embodiments, the structured data retriever agent module 506-3 can use a variety of different tools to retrieve structured data (e.g., structured data retrieval tool 508-2, filter tool 508-9, projection tool 508-10, grouping tool 508-11, sorting tool 508-12, and restriction tool 508-13, etc.). In some embodiments, once the structured data retriever agent module 506-3 has traversed the data model and retrieved (one or more) relevant types and / or subsets of the data model, the structured data retriever agent module 506-3 can then use this information and agent and / or tool output to construct a structured query specification that the structured data retriever agent module 506-3 can execute against one or more structured data stores to retrieve structured data records.

[0097] The type system retriever agent module 506-4 can function to retrieve types, data models, and / or subsets of data models. For example, a data model can include a variety of different types, and each type can describe a data field, an operation, and a function. Each type can represent a different object (e.g., a real-world object such as a machine or sensor in a factory, etc.), and each type can include a large language model context that provides context for a large language model. Types can be defined in a natural language format for efficient processing by a large language model.

[0098] In some embodiments, the type system is designed to be used by different computing systems, application developers, data scientists, operations personnel, and / or other users to build applications, develop and execute machine learning algorithms, and manage and monitor the status of jobs running on the type system (e.g., in some embodiments, the enterprise generative artificial intelligence system). The type system is a framework that enables systems, application developers, data scientists, and other users to communicate with each other efficiently using the same language. Thus, application developers can interact with the enterprise generative artificial intelligence system 402 in the same way as data scientists. For example, they can use the same types, the same methods, and the same features.

[0099] In some embodiments, the type system can abstract the complex infrastructure within enterprise generative AI system 402. In one example, developers may never need to write SQL, CQL, or some other query processing language to access data. When a user reads data, enterprise generative AI system 402 can generate the correct query for the underlying data store, submit the query to the database, and present the results back to the user in the form of an object or a collection of results.

[0100] In some embodiments, a type can be similar to a programming language class (e.g., a Java class) and describe data fields, operations, and functions (e.g., static functions) that can be called on the type or by one or more applications, but the type is not tied to any specific programming language. A type can be a definition of one or more complex objects that the system 402 can understand. For example, a type can represent a wide range of objects, such as a water pump. In addition to objects, types can also be used to model systems (e.g., computing clusters, key-value data stores, file systems, file stores, and enterprise data stores). In some embodiments, complex relationships such as "when which light bulbs are in which light fixtures" can be modeled as types.

[0101] The machine learning insight agent module 506-5 can function to obtain and / or process output from an artificial intelligence application (e.g., artificial intelligence application insights). For example, the machine learning insight module 506-5 can instruct the text processing tool 508-3 to perform a text processing task (e.g., converting an artificial intelligence application into natural language), instruct the image processing tool 508-4 to perform an image processing task (e.g., generating a natural language summary of an image output from an artificial intelligence application), instruct the time series tool 508-3 to summarize time series data (e.g., time series data output from an artificial intelligence application), and instruct the API tool 508-6 to perform an API call task (e.g., executing an API call to trigger or access an artificial intelligence application).

[0102] The time series processing agent 506 - 6 may function to obtain and / or process time series data, such as time series data output from various applications (e.g., artificial intelligence applications), machines, sensors, etc. The time series processing agent 506 - 6 may instruct and / or collaborate with the time series processing tool module 508 - 3 to obtain time series data from one or more artificial intelligence applications and / or other data sources.

[0103] The API agent module 506-7 can function to coordinate and manage communications with other applications. For example, the API agent module 506-7 can instruct the API tool module 508-6 to execute various API calls and then process the tool outputs (e.g., transform into natural language summaries).

[0104] The mathematical agent module 506-8 can function to determine whether the agent 506 or the large language model requires additional information to generate an answer or result. In some embodiments, the mathematical agent 506-8 can instruct the optimizer tool module 508-8 to perform various prescriptive parsing functions and mathematical optimizations to assist in the calculation of answers to various questions. For example, the orchestrator module 504 can use the mathematical agent module 506-8 to generate a plan and determine whether the orchestrator module 504 requires more information to generate a final result.

[0105] The visualization agent module 506-9 can function to generate one or more visualizations and / or graphical user interfaces, such as dashboards and charts. For example, the visualization agent module 506-9 can execute the visualization tool module 508-7 to generate a dashboard based on information retrieved by other agents and / or information output by other agents (e.g., a natural language summary of an associated tool output). The visualization agent module 506-9 can also function to generate summaries of visual elements (e.g., natural language summaries), such as charts, tables, and images.

[0106] The code generation agent module 506-10 can function to instruct the code generation tool module 508-14 to generate source code, machine code, and / or other computer code. For example, the code generation agent module 506-10 can be configured to determine what code is needed (e.g., to satisfy a query and create an application, etc.) and instruct the tool 508-14 to generate the code in a specific language or format.

[0107] In some embodiments, tools 508 are specific functions that an agent (e.g., agent 506, orchestrator module 504) can access or execute when attempting to complete (one or more) specified tasks (e.g., in a set of specified tasks of a plan determined by orchestrator module 504). Tools 508 can include software and / or hardware. Tools 508 can also include one or more machine learning models, but they can also include functionality without any machine learning models. In some embodiments, tools 508 do not include large language models, but in other embodiments, the tools can include large language models. In some embodiments, some or all of agents 506 and / or tools 508 can be manually configured (e.g., by a user). Agents 506 and tools 508 can also normalize data (e.g., to a common data format) before outputting the data.

[0108] Unstructured data retrieval tool 508-1 can function to retrieve unstructured data records from an unstructured data store. In some embodiments, agent 506-2 can use embeddings (e.g., vectors stored in vector storage 540) when retrieving information. For example, agent 506-2 can use similarity evaluation or search to find related data records based on k-nearest neighbors, where embeddings that are closer to each other are more likely to be related. Structured data retrieval tool 508-2 can function to access and retrieve structured data records from a structured data store (e.g., structured or modeled according to a data model). Structured data retrieval tool 508-2 can be executed by structured data retriever agent module 506-3).

[0109] The text processing tool module 508-3 can function to retrieve and / or transform text (e.g., from unstructured data records) and perform other text processing tasks (e.g., transforming text-based output of an artificial intelligence application into natural language). The image processing tool module 508-4 can function to perform image processing tasks (e.g., generating a natural language summary of an image). The time series processing tool module 508-5 can function to obtain and / or process time series data (e.g., output from an artificial intelligence application, a sensor, etc.). For example, the time series processing tool module 508-3 can be executed by one or more of the agents 506 to obtain and process time series data. The API tool module 508-6 can function to perform API call tasks (e.g., executing an API call to trigger or access an artificial intelligence application). For example, a different agent 506 can use the API tool module 506-8 each time the agent needs to access or trigger another application.

[0110] The visualization tool module 508-7 can function to generate one or more visualizations and / or graphical user interfaces, such as dashboards, etc. For example, the visualization tool module 508-7 can generate dashboards based on information retrieved by other agents and / or information output by other agents (e.g., a natural language summary of associated tool output). The filter tool module 508-9 can function to filter data records and / or types, etc. For example, the filter tool module 508-9 can filter projections (e.g., fields) identified by the projection tool module 508-10 as part of the structured data retrieval process. In various embodiments, the tools 508 can be executed in parallel or otherwise.

[0111] In some embodiments, the filter tool module 508-9 can identify implicit filters based on a query or other input, and those identified implicit filters can be used as part of a structured data retrieval process. For example, a query may include "When was the last time that premium towels sold out?" The filter tool module 508-9 can identify "premium towels" as a filter (e.g., based on an associated type description). The filter tool module 508-9 can also identify contextual datetime filters. For example, a query may include "How many systems were offline yesterday?" The filter tool module 508-9 can determine yesterday's date while taking into account time zones and other relevant data to generate accurate filters. In some embodiments, the filter tool module 508-9 can validate the identified filters before using them (e.g., as part of a structured data retrieval process).

[0112] The projection tool module 508-10 can function to identify and select fields (e.g., type fields, object fields) relevant to determining an answer to a query or other input. The grouping tool module 508-11 can function to group data (e.g., type and tool output, etc.), which can then be used (e.g., by the structured data retriever agent module 506-3 and / or the structured data retrieval tool module 508-2) to generate a structured query request.

[0113] The sorting tool module 508-12 can function to sort data (e.g., by type and tool output, etc.), which can then be used (e.g., by the structured data retriever agent module 506-3 and / or the structured data retrieval tool module 458-2) to generate a structured query request. The limiting tool module 508-13 can function to limit the output of the structured data retrieval process. For example, it can limit the number of retrieved data records, types, groups, and / or filters, etc.

[0114] The code generation tool module 508-14 can function to generate source code, machine code, and / or other computer code. For example, the code generation tool module 508-14 can be configured to generate and / or execute SQL queries and / or JAVA code, etc. The code generation tool module 508-14 can be used to facilitate query generation for agents, other tools, and large language models, etc. In some embodiments, the code generation tool module 508-15 can be configured to generate source code for an application or create an application.

[0115] The embedding generator module 512 can function to generate embeddings based on both structured and unstructured data records and / or segments. The embedding generator module 512 can be the same as the embedding generator module 128. The embedding generator module 512 can include one or more models (e.g., embedding models, deep learning models) that can convert and / or transform data records into vector representations in which vectors of semantically similar data records (e.g., the content of the data records) are close together in the vector space. This can facilitate retrieval operations using the agent 506 and the tool 508.

[0116] In some embodiments, the embedding generator module 512 can generate embeddings using one or more embedding models (e.g., an implementation of the ColBERT embedding model). Embeddings can include digital representations of unstructured and / or structured data records that capture the semantic or contextual meaning of the data records. For example, an embedding can be represented by one or more vectors. The embeddings can be used when retrieving data records and performing similarity assessments or other aspects of the retrieval operation. The embeddings can be stored in an embedding index (e.g., a vector data store 540). In some embodiments, the vector store 540 is a type of database that is specifically optimized for storing embeddings and retrieving embeddings using a similarity heuristic (e.g., an approximate nearest neighbor (ANN) algorithm) that can be implemented by the agent 506 and / or the tool 508. In one example, the vector store 540 can include an implementation of the FAISS vector store.

[0117] The chunking module 510 may be the same as the chunking module 124 .

[0118] In some embodiments, the embedding generator module 512 generates an enhanced embedding. For example, the chunking module 510 can generate an enhanced embedding based on contextual information, data records, and / or data record segments. The enhanced embedding can include a vector value based on the embedding vector and the contextual information. In some embodiments, the enhanced embedding includes the embedding vector value and contextual metadata including the contextual information. The enhanced embedding can be indexed in an enhanced embedding data store (e.g., vector data store 540). The agent 506 and / or the tool 508 can retrieve unstructured and / or structured data records based on the enhanced embedding. In some embodiments, the contextual information can be included and / or represented in the contextual metadata, and / or the contextual metadata can be generated from the contextual information.

[0119] The crawler module 514 can function to scan and / or crawl different data sources (e.g., enterprise data sources, external data sources) across different domains. The crawler module 514 can be the same as the crawler module 122. The crawler module 514 can identify existing data records, new data records, and / or updated data records. The crawler module 514 can notify the chunking module 510 of new data records and updated data records, and the chunking module 510 can chunk the data records. In some embodiments, the crawler module 514 can function and / or trigger operations periodically, on demand, and / or in real time.

[0120] The enterprise generative artificial intelligence system 402 can perform some or all of the functionality described herein periodically (e.g., in batches), on-demand, and / or in real-time. For example, the system can trigger the intelligent crawling and indexing described herein periodically, on-demand, manually, and / or automatically. In some implementations, subsequent crawling and indexing operations can incorporate only changes (e.g., "deltas") relative to previous crawling and indexing operations.

[0121] The understanding module 516 can function to process the input to determine a result (e.g., an "answer"), determine the root cause of the result, and determine whether the understanding module 516 needs more information to determine the result. The understanding module 516 can output information (e.g., results or additional queries) in a natural language format or a machine language format. In some implementations, features of one or more models of the understanding module define conditions or functions for determining whether more information is needed to satisfy the initial input or whether sufficient information exists to satisfy the initial input.

[0122] In some embodiments, the understanding module 516 includes one or more large language models. The large language model can be configured to generate and process context and other information described herein. The understanding module 516 can also include other language models that pre-process the input (e.g., user query) before the input is provided to the agent for processing. The understanding module 516 can also include one or more large language models that process the outputs from other models and modules (e.g., the model of the agent 506). The understanding module 516 can also include another large language model that is used to process the answer from a large language model into a format that is more consistent with the final answer that can be transmitted to various users and / or systems (e.g., the user or system that provided the initial query or other intended recipient of the answer).

[0123] For example, the understanding module 516 can format answers based on various viewpoints. The viewpoints can be based on the type of user (e.g., human or machine), the user's role (e.g., data scientist, engineer, director, etc.), and access permissions. Thus, the viewpoints enable the understanding module 516 to generate and provide answers that are specifically tailored to the recipient. The understanding module 516 can also notify the user and the system if it cannot find an answer (e.g., as opposed to presenting an answer that is likely to be erroneous or biased).

[0124] In some implementations, features of one or more large language models of the understanding module 516 define conditions or functions for determining whether more information is needed to satisfy the initial input or whether sufficient information exists to satisfy the initial input. The large language model of the understanding module 516 may also define a stopping condition indicating a stopping threshold condition that indicates a maximum number of iterations that can be performed before the iterative process terminates.

[0125] In some embodiments, the understanding module 516 can generate and store root causes and context (e.g., in the data store 910). The root cause can be an inference used by the understanding module 516 to determine an output (e.g., a natural language output, an indication that it needs more information, an indication that it can satisfy the initial input). The understanding module 516 can generate the context based on the root cause. In some implementations, the context includes a concatenation and / or annotation of one or more segments of the data record, and / or embeddings associated therewith and a mapping of the concatenation and / or annotations. For example, the mapping can indicate relationships between different segments and / or weighted or relative values ​​associated with different segments, etc. The root cause and / or context can be included in the hints provided to the large language model.

[0126] In some embodiments, the understanding module 516 includes a query and root cause generator that generates queries or other inputs for a model (e.g., a large language model, other machine learning model) and / or generates and stores root causes and context (e.g., in the data store 560). The query and root cause generator can function to process, extract, and / or transform different types of data (e.g., text, database tables, images, videos, and / or code, etc.). For example, the query and root cause generator can accept a database table as input and transform it into natural language describing the database table, which can then be provided to one or more other models (e.g., a large language model) of the understanding module 516, which can then process the transformed input into an “answer” or otherwise satisfy the query. In some implementations, the query and root cause generator includes a model that can convert (or transform) images, database tables, and / or other non-textual input into a textual format (e.g., natural language). It will be understood that although queries are used in various examples herein, other types of input (e.g., instruction sets) can be processed in the same or similar manner as described for queries.

[0127] In some embodiments, the understanding module 516 can use different models for different domains. For example, different domains can correspond to different industries (e.g., aerospace, defense), different technical environments (e.g., on-premises, air-gapped, cloud-native), and / or different enterprises or organizations, etc. Thus, the understanding module 516 can use a specific model (e.g., a data model and / or a large language model) for a specific domain (e.g., a data model that describes the properties and relationships of aerospace objects and a large language model trained on an aerospace-specific dataset), and use another data model and / or large language model for another domain (e.g., a data model that describes the properties and relationships of defense-specific objects and a large language model trained on a defense-specific dataset), and so on.

[0128] In some embodiments, orchestrator module 504 includes some or all of the functionality and / or structure of comprehension module 516 and / or 906, described further below. Similarly, in some embodiments, comprehension module 516 can include some or all of the functionality and / or structure of orchestrator module 504.

[0129] In some embodiments, the understanding module 516 can operate to generate large language model prompts (or simply referred to as prompts) and prompt templates. For example, the understanding module 516 can generate a prompt template for processing an initial input, a prompt template for processing iterative inputs (i.e., inputs received during an iterative process after processing the initial input), and another prompt template for an output result phase (i.e., when the understanding module 516 has determined that it has sufficient information and / or meets a stop condition). The understanding module 516 can modify appropriate prompt templates according to the phase of the iterative process. For example, the prompt templates can be modified to generate prompts that include root causes and contexts, which can inform subsequent iterations.

[0130] The enterprise access control module 518 can operate to provide enterprise access control (e.g., layers and / or protocols) for the enterprise generative AI system 402, associated systems (e.g., enterprise systems), and / or environments (e.g., enterprise information environments). The enterprise access control module 518 can provide functionality for enforcing access control policies for generated results (e.g., preventing the orchestrator module 504 and / or the understanding module 516 from generating results that include sensitive information) and / or filtering results that have been generated before providing the final result.

[0131] In some implementations, the enterprise access control module 518 can (e.g., using an access control list) evaluate whether a user is authorized to access all or only a portion of a result (e.g., an answer). For example, a user can provide a query associated with a first department or subunit of an organization. Members of that department or subunit can be restricted from accessing certain data, data types, data models, or other aspects of the data domain to be searched. In the case where the initial result includes data that the user has restricted access to, the enterprise access control module 518 can determine how to handle such restricted data, such as completely omitting the restricted data, omitting the restricted data but indicating that the result includes data that the user has restricted access to, or providing information related to all of the initial results. In an example of completely omitting the restricted data, a final result set can be returned for presentation to the user, where the final result set does not inform the user that a portion of the initial result has been omitted. In an example of omitting the restricted data but providing the user with an indication that there is restricted data, the final result can include only those results that the user is authorized to access, but can include information indicating that there were X initial results but only Y results were output, where Y < X. In the third example above, all results, including those that the user has restricted access to, can be output to the user.

[0132] Additionally or alternatively, the enterprise access control module 518 may communicate with one or more other modules to obtain information that may be used to enforce access permissions / restrictions in conjunction with performing a retrieval operation rather than for controlling the presentation of results to a user. For example, the enterprise access control module 518 may restrict the data sources to which a retrieval operation is applied, such as not applying the retrieval operation to portions of the data source to which the user is denied access and applying the retrieval operation to portions of the data source to which the user is permitted access, etc. Note that the exemplary techniques for enforcing access restrictions described above have been provided for purposes of illustration and not by way of limitation, and it should be understood that modules operating in accordance with embodiments of the present disclosure may implement other techniques for presenting results via an interface based on access restrictions.

[0133] In some embodiments, to facilitate enforcing access restrictions in conjunction with searches conducted by enterprise generative artificial intelligence system 402, enterprise access control module 518 may store information associated with access restrictions or permissions for each user. To retrieve relevant restriction data for a user, enterprise access control module 518 may receive information identifying the user in conjunction with an input or when the user logs into the system on which enterprise access control module 518 is executing. Enterprise access control module 518 may use the information identifying the user to retrieve appropriate restriction data to support enforcing access restrictions in conjunction with enterprise searches. In some embodiments, enterprise access control module 518 may include credential management functionality of a model-driven architecture deployed with enterprise generative artificial intelligence system 402, or may be a remote credential management system communicatively coupled to enterprise generative artificial intelligence system 402 via a network.

[0134] The AI ​​traceability module 516 can function to provide traceability and / or explainability of answers generated by the enterprise generative AI system 402. For example, the AI ​​traceability module 516 can indicate the portion of the data record used to generate the answer and its associated data source. The AI ​​traceability module 516 can also function to validate the large language model output. For example, the AI ​​traceability module 516 can automatically and / or on-demand provide source references to validate or verify the large language model output. The AI ​​traceability module 516 can also determine the compatibility of different sources (e.g., data records, passages) used to generate the large language model output. For example, the AI ​​traceability module 516 can identify data records that contradict each other (e.g., one data record indicates that John Doe is an employee of Acme Company, while another data record indicates that John Doe works for a different company) and provide a notification that the output is a contradiction generated based on the conflicting information.

[0135] The parallelization module 522 can function to control the parallelization of the various systems, modules, agents, models, and processes described herein. For example, the parallelization module 522 can cause the parallel execution of different agents and / or orchestrators. The parallelization module 522 can be controlled by the orchestrator module 504.

[0136] The model generation module 524 can function to obtain, generate and / or modify some or all of the different types of models described herein (e.g., machine learning models, large language models, data models). In some implementations, the model generation module 524 can use various machine learning techniques or algorithms to generate models. As used herein, artificial intelligence and / or machine learning can include Bayesian algorithms and / or models, deep learning algorithms and / or models (e.g., artificial neural networks, convolutional neural networks), gap analysis algorithms and / or models, supervised learning techniques and / or models, unsupervised learning algorithms and / or models, semi-supervised learning techniques and / or models, random forest algorithms and / or models, similarity learning and / or distance algorithms, generative artificial intelligence algorithms and models, clustering algorithms and / or models, transformer-based algorithms and / or models, neural network transformer-based machine learning algorithms and / or models, and / or reinforcement learning algorithms and / or models, etc. Algorithms can be used to generate corresponding models. For example, an algorithm can be executed on a data set (e.g., a domain-specific data set, an enterprise data set) to generate and / or output a corresponding model.

[0137] In some embodiments, a large language model is a deep learning model (e.g., generated by a deep learning algorithm) that can recognize, summarize, translate, predict, and / or generate text and other content based on knowledge obtained from a large-scale data set. Large language models may include transformer-based models. Large language models may include Google's BERT, OpenAI's GPT-3, and Microsoft's Transformer. Large language models can process large amounts of data, thereby improving the accuracy of prediction and classification tasks. Large language models can use this information to learn patterns and relationships, which can help them make improved predictions and groupings relative to other machine learning models. Large language models may include artificial neural network transformers pre-trained using supervised and / or semi-supervised learning techniques. In some embodiments, large language models include deep learning models specifically for text generation. In some embodiments, large language models may be characterized by a large number of parameters (e.g., tens of billions or hundreds of billions of parameters) and a large text corpus for training them.

[0138] Although the systems and processes described herein use large language models, it will be understood that other embodiments may use different types of machine learning models instead of or in addition to large language models. For example, orchestrator 504 may use a deep learning model specifically designed to receive non-natural language input (e.g., images, videos, audio) and provide natural language output (e.g., summaries) and / or other types of output (e.g., video summaries).

[0139] Model deployment module 526 can function to deploy some or all of the different types of models described herein. In some implementations, model deployment module 526 can deploy models before or after deployment of an enterprise generative artificial intelligence system. For example, model deployment module 526 can collaborate with model optimization module 528 to exchange or otherwise modify a large language model of an enterprise generative artificial intelligence system.

[0140] In some implementations, the model registry 550 can store various models (e.g., machine learning models, large language models, data models) and / or model configurations. Models can be trained on general datasets and / or domain-specific datasets. For example, the model registry can store different configurations of various large language models (e.g., which can be deployed or exchanged within the enterprise generative artificial intelligence system 402). In some embodiments, each model can be associated with an embedded value or an enhanced embedded value to facilitate retrieval operations (e.g., in the same or similar manner as data record retrieval).

[0141] Model optimization module 528 can function to enable tuning and learning using the modules (e.g., understanding module 212) and / or models (e.g., machine learning models, large language models) described herein. For example, model optimization module 528 can tune understanding module 516 and / or orchestrator module 504 (and / or its models) based on tracking user interactions within the system, capturing explicit and / or implicit feedback (e.g., through a training user interface), and / or the like. In some example implementations, model optimization module 528 can use reinforcement learning to accelerate knowledge base bootstrapping. Reinforcement learning can be used to explicitly bootstrap various systems (e.g., enterprise generative artificial intelligence system 402) by instrumenting, for example, time spent and / or results clicked. Example aspects of model optimization module 528 include an innovative learning framework that can bootstrap models for different enterprise environments. Example aspects of model optimization module 528 can include an innovative learning framework that can bootstrap models for different enterprise environments.

[0142] In some embodiments, reinforcement learning is a machine learning training method based on rewarding desired behavior and / or punishing undesirable behavior. Typically, a reinforcement learning agent is able to perceive and interpret its environment, take actions, and learn through trial and error. Reinforcement learning uses algorithms and models to determine the optimal behavior in the environment to obtain the maximum reward. This optimal behavior is learned by interacting with the environment and observing how it responds. In the absence of a supervisor, the learner must independently discover the sequence of actions that maximizes the reward. This discovery process is similar to trial and error search. The quality of the actions is measured not only by the immediate rewards they return, but also by the delayed rewards they may obtain. Because it can learn actions that lead to ultimate success in an unseen environment without the help of a supervisor, reinforcement learning is a very powerful algorithm. ColBERT is an example retriever model that enables scalable BERT-based search on large text collections (e.g., within tens of milliseconds). ColBERT uses a late interaction architecture that uses BERT to independently encode queries and documents, and then adopts a "cheap" but powerful interaction step to model their fine-grained similarity. In addition to reducing the cost of re-ranking documents retrieved by traditional models, ColBERT’s pruning-friendly interaction mechanism also enables end-to-end retrieval directly from large document collections by leveraging vector similarity indices.

[0143] In some embodiments, the model optimization module 528 can retrain the model (e.g., the transformer-based natural language machine learning model) periodically, on demand, and / or in real time. In some example implementations, corresponding candidate models (e.g., candidate transformer-based natural language machine learning models) can be trained based on user selections, and the model optimization module 528 can replace some or all of the models with one or more candidate models that have been trained on the received user selections.

[0144] In some embodiments, in addition to before or after runtime, model optimization module 528 can also replace the models of the enterprise generative artificial intelligence system at runtime or during runtime. For example, orchestrator module 504, understanding module 516 and / or agent 506 can use a specific set of machine learning models for one domain and use other models for a different domain. The model exchange module can select and use the appropriate model for a given domain. This can even occur during an iterative process. For example, when a new query is generated by understanding module 516, the domain may change, which may trigger the model exchange module to select and deploy a different model appropriate for that domain.

[0145] In some embodiments, the model optimization module 528 can train the generative artificial intelligence model to develop different types of responses (e.g., best results, ranked results, smart cards, chatbots, and / or new content generation, etc.).

[0146] In some embodiments, the model optimization module 528 can retrain models (e.g., large language models) periodically, on demand, and / or in real time. In some example implementations, corresponding candidate models can be trained based on user selections, and the system can replace some or all of the models with one or more candidate models that have been trained on received user selections.

[0147] The interface module 530 can function to receive input (e.g., complex input) from a user and / or system. The interface module 530 can also generate and / or transmit output. The input can include system input and user input. For example, the input can include an instruction set, a query, a natural language input or other human-readable input, and / or a machine-readable input, etc. Similarly, the output can also include system output and human-readable output. In some embodiments, the input (e.g., a request, a query) can be input in various natural forms for human interaction (e.g., a basic text box interface, image processing, and / or voice activation, etc.) and processed to quickly find relevant and responsive information.

[0148] In some embodiments, interface module 530 may function to generate a graphical user interface component (e.g., a server-side graphical user interface component) that can be rendered as a complete graphical user interface on enterprise generative artificial intelligence system 402 and / or other systems. For example, interface module 530 may function to present an interactive graphical user interface for displaying and receiving information.

[0149] The communication module 532 may function to send requests, transmit and receive communications, and / or otherwise provide communications with one or more of the systems, modules, engines, layers, devices, data stores, and / or other components described herein. In certain implementations, the communication module 532 may function to encrypt and decrypt communications. The communication module 532 may function to send requests to and receive data from one or more systems via a network or portion of a network (e.g., the communication network 408). In certain implementations, the communication module 532 may send requests and receive data via a connection, all or part of which may be a wireless connection. The communication module 532 may request and receive messages and / or other communications from associated systems, modules, and / or layers, etc. Communications may be stored in the enterprise generative artificial intelligence system data store 570.

[0150] In some embodiments, the configuration, coordination, and collaboration of orchestrator module 504, agents 506, tools 508, and / or other modules of enterprise generative AI system 402 (e.g., understanding module 516) enable enterprise generative AI system 402 to provide a multi-hop architecture that enables complex reasoning across multiple agents 506, tools 508, and data sources (e.g., vector data stores, feature data stores, data models, enterprise data stores, unstructured data sources, structured data sources, etc.). In various embodiments, some or all of the modules of enterprise generative AI system 402 can be configured manually (e.g., by a user) and / or automatically (e.g., without requiring user input). For example, large language model hints can be configured, tool 508 descriptions can be configured (e.g., for more efficient utilization by agents 506 and orchestrator module 504), and a maximum number of hops or iterations can be configured. In one example, orchestrator module 504 receives a query from a user, and orchestrator module 504 determines a plan for answering the query and selects agents 506 and / or tools 508 to perform a set of prescribed tasks that formulate the plan to answer the query. Agents 506 and / or tools 508 perform the set of prescribed tasks, and orchestrator module 504 observes the results. Orchestrator module 504 determines whether to submit a final answer or whether orchestrator module 504 requires more information. If orchestrator module 504 has sufficient information, it can generate and / or provide a final answer. Otherwise, orchestrator module 504 can create another set of prescribed tasks, and the process can continue until orchestrator module 504 has sufficient information to answer or a stopping condition (e.g., a maximum number of hops) is met.

[0151] Figure 6 A flowchart 600 is depicted of an example generative artificial intelligence unstructured data and structured data retrieval process. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram, as well as other flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that, where applicable, some or all of the steps may be repeated, reorganized for parallel execution, and / or reordered. In addition, for clarity, some steps may have been removed to avoid providing too much information, and some steps may have been included but could have been removed for clarity.

[0152] In step 602, a user provides a query that is received by orchestrator 604. Although a user query is shown here, it will be understood that other inputs (e.g., machine-readable input received from other systems) may be used instead. The orchestrator module may pre-process the user query. For example, orchestrator 604 may translate the machine-readable input into a natural language format, translate French language input into English language input, and so on.

[0153] In step 606, the orchestrator generates a query 606 based on the user query and selects various tools to process the orchestrator query 606. In step 608, the orchestrator 604 selects an unstructured data retrieval tool 610 and a structured query generator tool 612. The unstructured query tool 610 may query a data store 616 (e.g., a vector store) for relevant unstructured data records (step 614). The unstructured data records may be provided to another model 618 (e.g., a large language model of the orchestrator 604), which may generate a summary 620 of the retrieved unstructured data records. For example, the large language model of the orchestrator 604 may not be able to handle large data inputs, so the model 618 may generate a summary in a natural language format that can be efficiently and accurately processed by the orchestrator 604.

[0154] In step 622, structured query builder tool 612 queries type system 624 for relevant data. Query response 626 may identify one or more database tables and / or API calls from step 626, retrieve the database tables and / or types, and execute the API calls (e.g., for artificial intelligence applications). Visualization tool 628 may provide a visual summary 630 of the retrieved database tables and API call execution results. For example, visual summary 630 may include graphical elements (e.g., graphs, charts, dashboards) and / or natural language summaries of such graphical elements. In step 632, the orchestrator may generate a final result based on summary 620 (i.e., summaries based on unstructured data) and summary 630 (i.e., summaries based on structured data retrieval). The final result may include a natural language summary based on the user's perspective. The visualization may be three-dimensional (3D) and include, for example, interactive elements associated with deterministic outputs that enable execution of instructions (e.g., transmissions, control system commands, etc.), drilling down into traceability, activating application features, and the like.

[0155] In some embodiments, Figure 6The orchestrators, tools, and / or data stores described in

[0065] include some or all of the functionality of the orchestrators, agents, tools, and / or data stores described elsewhere herein. Thus, for example, orchestrator 604 may include some or all of the functionality of orchestrator 504, unstructured query tool 610 may include some or all of the functionality of unstructured data retriever agent module 506-2 and / or unstructured data retriever tool module 506-1, and so on.

[0156] Figure 7 Diagram 700 depicts an example logic flow of an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. In step 702, an orchestrator module (e.g., orchestrator module 504) receives input (e.g., complex input) and generates a plan and a corresponding set of prescribed tasks. The orchestrator generates several subqueries from the input, the plan, and / or the set of prescribed tasks. The orchestrator coordinates an unstructured data agent (e.g., unstructured data retriever agent module 506-2) to process a first subquery that may be based on a first portion of the set of prescribed tasks, coordinates a structured data agent (e.g., structured data retriever agent module 506-3) to process a second subquery that may be based on a second portion of the set of prescribed tasks, coordinates a time series processing agent module (e.g., time series processing agent module 506-6) to process a third subquery that may be based on a third portion of the set of prescribed tasks, and coordinates a visualization tool (e.g., visualization agent module 506-9 and / or visualization tool module 507-6) to generate a graphical representation of the final answer generated by the orchestrator. In some embodiments, the orchestrator may directly coordinate Figure 7 Other tools shown and / or indirectly coordinate other tools. For example, the orchestrator can directly coordinate (e.g., instruct) the agents, which can then directly coordinate their respective tools.

[0157] Steps 704-712 depict the generative artificial intelligence unstructured data retrieval and answering process (e.g., to answer the first sub-query). More specifically, in step 704, the unstructured data agent selects an unstructured data retrieval tool (e.g., unstructured data retrieval tool 508-1) to retrieve unstructured data records. In step 704, the unstructured data retrieval tool retrieves the unstructured data records from the vector data store 706 (e.g., using a similarity heuristic search, such as an ANN algorithm, etc.). In step 710, a large language model (e.g., a large language model of the unstructured data agent) extracts relevant paragraphs from the retrieved data records. In step 712, the large language model (e.g., a large language model of the unstructured data agent and / or the orchestrator) generates an answer to the first sub-query.

[0158] Steps 714-738 depict the generative artificial intelligence structured data retrieval and answering process (e.g., to answer the second sub-query). More specifically, in step 714, the structured data agent selects a type system retriever agent (e.g., type system retriever agent module 506-4). In step 716, in some embodiments, the type system retriever agent searches for types of data models in one or more data models in vector storage 718, and in step 720, the type system retriever agent (e.g., the type system agent's large language model) selects types of data models that are relevant to answering the initial query and / or the second sub-query. The retrieved types may include subsets (or sub-models) or data models that are relevant to the initial query and / or the second sub-query.

[0159] In step 720, a projection tool (e.g., projection tool module 508-10) selects a projection (e.g., a specific field of a relevant type) that is relevant to answering the initial query and / or the second subquery. The projection may be selected based on a type field document describing the type field from the data store 724. Based on the selected projection, the structured data agent may select a different tool to generate a structured data retrieval specification query (step 722). Figure 7 In the example of , the structured data agent selects a filter tool (e.g., filter 508-9) to filter the selected type based on the selected projection (step 726), selects a grouping tool (e.g., grouping tool module 508-11) to group the types (e.g., filtered types) and / or projections (step 728), selects a restriction tool (e.g., restriction tool module 508-13) to restrict the structured data results (step 730), and selects a sorting tool (e.g., sorting tool 508-12) to sort the types and / or fields so that the structured data results are provided in a specific order (step 734). In step 736, the structured data agent creates a structured data retrieval specification query based on the tool output of steps 726-734 and executes the query against the structured data store. In step 738, the large language model (e.g., the large language model of the structured data agent and / or the orchestrator) generates an answer (e.g., to the second sub-query) based on the results of the query.

[0160] Steps 740-760 depict the generative artificial intelligence time series data retrieval and answering process. The generative artificial intelligence time series data retrieval and answering process can function in the same or similar manner as the generative artificial intelligence structured data retrieval and answering process, except that time series data is used instead of structured data. For example, a time series agent (e.g., time series agent 506-6) can be used instead of a structured data agent, and a time series processing tool (e.g., time series processing tool 508-5) can be used with a filter tool to construct a canonical query that can be used to retrieve time series data records (e.g., artificial intelligence application output) to generate an answer (e.g., an answer to the third subquery).

[0161] In step 762, the orchestrator may use a large language model that receives answers 712, 738, and 760 as input and generates results based on these inputs. A visualization tool may generate a graphical representation of the results (e.g., a graph, chart, dashboard, etc.).

[0162] Figure 8A A flowchart 800 depicts an example iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 802, a user query is provided to a retrieval model (e.g., a retriever module of a retrieval agent module). In step 804, the retrieval model receives the query and performs a similar search (e.g., an ANN-based search) of vector storage 806. In step 810, the retrieved information is returned to the retriever model and provided to a large language model. In step 812, the large language model (e.g., the large language model used in step 810 and / or a different large language model) determines whether additional information is needed to answer the user query. If more information is needed, steps 804-812 can be iteratively repeated using updated large language model prompts until the large language model has sufficient information to answer the query or a stopping condition is met (e.g., a maximum number of iterations have been performed). In step 814, an answer is generated (e.g., a final result if there is enough information for the large language model to determine the answer, or "I don't know" if a stopping condition is met before sufficient information can be received). The final result 814 may also include the rationale used by the large language model to generate the answer.

[0163] Figure 8BA flowchart 830 depicts an example non-iterative generative artificial intelligence process using unstructured data, according to some embodiments. This example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 832, a query is received. Query 832 is executed against vector store 834, and relevant paragraphs 836 are retrieved. In some embodiments, user query 832 is pre-processed (e.g., by an orchestrator) before being applied to vector store 834 to retrieve paragraphs 836. For example, query 832 can be translated, transformed, etc. Because vector stores can have difficulty handling complex input, the system can generate a new query or multiple shorter queries from user query 832 that vector store 834 can handle efficiently and accurately. Query 832 and paragraphs 836 are provided to a large language model 838, which can create extractions 840 (e.g., summaries of the paragraphs) for each paragraph. The extractions are combined (e.g., concatenated) in step 842 and provided to large language model 844 along with query 832. In some embodiments, the extraction step is optional, and instead of extraction, the paragraphs can be concatenated and provided to a large language model 844. The large language model 844 can generate a final response based on the query 832 and the combined extraction 842. In some embodiments, the large language model 844 can post-process the results (e.g., using an orchestrator) before providing them to the user. For example, they can be translated based on the viewpoint, formatted, include references and attributes, etc.

[0164] Figure 8C A flowchart 860 is depicted of an example non-iterative generative artificial intelligence process using unstructured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 862, a query is received. For example, the query can be "How much wine do they produce?" This query is difficult for a traditional large language model to process and will typically cause the large language model to hallucinate because it is unclear how to process the "they" in query 862. To address this issue, the enterprise generative artificial intelligence system can use context 864 to generate improved queries. For example, a previous conversation 864 (e.g., as part of a chat with a chatbot) may have included a conversation about France. The system can provide France as contextual information 864 to generate a new query 866, such as "How much wine does France produce?" This can prevent the large language model from hallucinating and allow the large language model to provide an accurate and reliable final result 878.

[0165] More specifically, the enterprise generative artificial intelligence system can generate a rewritten query 866 that can be executed against vector store 868 to retrieve paragraph 870. In some embodiments, rewritten query 866 can be preprocessed (e.g., by an orchestrator) before being applied to vector store 868. For example, rewritten query 866 can be translated, transformed, and the like. Because vector stores have difficulty handling complex input, the system can generate a new query or multiple shorter queries from rewritten query 866 that vector store 868 can handle efficiently and accurately. In some embodiments, this preprocessing can be performed when generating the rewritten query (e.g., rewriting the query includes a preprocessing step).

[0166] The rewritten query 866 and the paragraphs 870 are provided to a large language model 869, which can create extractions 872 (e.g., summaries of the paragraphs) for each paragraph. The extractions 872 are combined (e.g., concatenated) in step 874 and provided to a large language model 876 along with the rewritten query 866. The enterprise generative artificial intelligence system can use the combined extractions 874 to generate root causes for determining a final response 878. For example, the large language model 876 can generate a final response 878 based on the root causes and / or present the root causes (or a summary of the root causes) with the final response 878 (e.g., for citation or attribution purposes).

[0167] In some embodiments, the extraction step is optional, and instead of extracting or combining extractions, the paragraphs can be concatenated and provided to a large language model 876. The large language model 844 can generate a final response 878 based on the rewritten query 866 and the combined extraction 874. In some embodiments, the large language model 876 can post-process the final result 878 before providing it to the user (e.g., using an orchestrator). For example, it can be translated based on the viewpoint, formatted, include references and attributes, etc.

[0168] Figure 9 A flowchart 900 is depicted of an example iterative generative artificial intelligence process using unstructured data according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In this flowchart and / or sequence diagram and other flowcharts and / or other sequence diagrams, the flowchart illustrates a sequence of steps by way of example. Figure 9In the example of , an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) includes one or more retrieval modules 904 and one or more understanding modules 906. For example, retrieval module 904 may include one or more large language models, and understanding module 906 may include one or more other large language models. These modules and the iterative interaction (e.g., communication) between these modules can allow the enterprise generative artificial intelligence system to achieve the technical features and technical benefits discussed herein.

[0169] exist Figure 9 In the example of , an enterprise generative artificial intelligence system may receive initial input 902 from a user or another system. For example, an orchestrator module 903 may receive input 902. The enterprise generative artificial intelligence system may provide this input to a retrieval module 904 (e.g., corresponding to one or more of agents 506 and / or tools 508), which may then extract and "retrieve" information from embedding storage 908. For example, retrieval module 904 may retrieve passages based on the input by using one or more similarity heuristics (e.g., an approximate nearest neighbor (ANN) algorithm) executed on embedding storage 908 (e.g., one or more vector stores) to retrieve passages (or data records) related to the input.

[0170] The enterprise generative artificial intelligence system can use this retrieved information to generate an initial prompt for the understanding module 906. The understanding module 906 can process the initial prompt and determine whether it has enough information to meet the criteria (e.g., answer the question) based on the initial input. See, for example, step 907. If it has enough information to meet the initial input, the understanding module can then provide the results to a recipient (see, for example, step 913), such as the user or system that provided the initial input. However, if the understanding module 906 determines that it does not have enough information to meet the criteria based on the initial input, it can further synthesize information through an iterative process that provides the core benefits of the system.

[0171] There are many reasons why understanding module 906 may need additional information. For example, traditional systems use a single-pass process that only processes a portion of a complex input. Enterprise generative artificial intelligence systems address this problem by triggering subsequent iterations to solve for the remaining portion of the complex input and including context to further refine the process.

[0172] More specifically, if the understanding module 906 determines that it needs additional information to satisfy the initial input, it can generate context-specific data (or simply "context") that will inform future iterations of the process and help the system satisfy the initial input more efficiently and accurately. The context is based on the root cause used by the understanding module 906 when it is processing the query (or other input). For example, the understanding module 906 can receive segments of information retrieved by the retrieval module 904. The segments can be, for example, paragraphs of (one or more) data records, and the segments can be associated with embeddings from the embedding data store 908 that facilitate the processing of the understanding module 906. The query and root cause generator 912 of the understanding module 906 can process this information and generate a root cause for why the result produced by the generator 912 was produced. The root cause can be stored by the enterprise generative artificial intelligence system in the historical root cause data store 910 and provide a basis for subsequent iterations of the context.

[0173] More specifically, subsequent iterations may include the understanding module 906 generating a new query, request, or other output, which is then passed back to the retrieval module. The retrieval module 904 may process the new query and retrieve additional information. The system then generates a new prompt based on the additional information and context. The understanding module 906 may process the new prompt and again determine whether it requires additional information. If it requires additional information, the enterprise generative artificial intelligence system may repeat (e.g., iterate) the process until the understanding module 906 can meet the criteria based on the initial input, at which point the understanding module 906 may generate an output 914 (e.g., "answer" or "I don't know"). For example, if no relevant paragraphs are generated or retrieved (e.g., by applying the rules of the understanding module 906) and / or not enough relevant paragraphs are generated, retrieved, and / or extracted, then the answer "I don't know" is generated. The understanding module 906 can prevent hallucinations and improve the performance of "I don't know" queries while saving calls to a model (e.g., a large language model).

[0174] In some embodiments, whether sufficient information exists can be determined and / or correlated based on the number of retrieved paragraphs, but relevant information has not been extracted (e.g., by comprehension module 906). For example, a threshold number or percentage of retrieved paragraphs (e.g., a particular number or percentage of retrieved paragraphs) may need to be met for relevant information to be extracted for business comprehension module 906 to determine that it has sufficient information to answer the query. In another example, a threshold number or percentage of retrieved paragraphs (e.g., 4 paragraphs or 80% of the retrieved paragraphs) that do not extract relevant information may cause business comprehension module 906 to determine that it does not have sufficient information to answer the query.

[0175] The enterprise generative artificial intelligence system may also implement supervisory functions, such as stopping conditions that prevent the system from hallucinating or otherwise providing incorrect answers. Stopping conditions may also prevent the system from executing an infinite iterative loop. In one example, the enterprise generative artificial intelligence system may limit the number of iterations that can be performed before the understanding module 906 will provide an output result or indicate that an output result cannot be found. Users may also receive feedback 916 that may be stored in a feedback data store 918. In some embodiments, the enterprise generative artificial intelligence system may use feedback to improve the accuracy and / or reliability of the system. As discussed elsewhere herein, it will be understood that in some embodiments, the functionality of the understanding module may be included within the orchestrator.

[0176] Figure 10 Flowchart 1000 depicts an example generative artificial intelligence process using unstructured and structured data, according to some embodiments. The example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 1002, the enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) processes input (e.g., complex input) using an orchestrator (e.g., orchestrator module 504). The orchestrator can include one or more first large language models, first machine learning models, and / or first functions (e.g., supervised functions).

[0177] In step 1004, the enterprise generative artificial intelligence system utilizes an orchestrator to select a first agent from a plurality of different agents based on the processed input. In some embodiments, the first agent (e.g., an unstructured data retrieval agent) is generated by a retrieval agent module (e.g., retrieval agent module 506-1) and / or an unstructured data retriever agent module 506-2. In some embodiments, the first agent includes one or more second large language models (e.g., a large language model different from the first large language model).

[0178] In step 1006, the enterprise generative artificial intelligence system uses a first agent to retrieve one or more first data records from an unstructured data set. The first data record can be an unstructured data record retrieved from one or more vector data stores and / or based on one or more vector data stores. For example, the first agent can implement Figure 9 In some embodiments, the first agent uses (e.g., executes, accesses, and / or controls) one or more tools (e.g., unstructured data retrieval tool module 508-1) to retrieve unstructured data records. In some embodiments, the first agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process (e.g., Figure 8A and Figure 9 ) and / or non-iterative generative AI processes (e.g., as Figure 7 and Figures 8B to 8C ) to retrieve one or more first data records from an unstructured dataset. In one example, the first agent instructs one or more unstructured data retrieval tools based on embeddings in the vector storage. For example, the first agent and / or tool can use an artificial intelligence-based similarity search (e.g., an ANN algorithm) to identify and retrieve paragraphs with similar embedding values ​​(e.g., closest in vector space based on one or more thresholds).

[0179] In step 1008, the enterprise generative artificial intelligence system utilizes the orchestrator to select a second agent from the plurality of different agents based on the processed input. In some embodiments, the second agent (e.g., an unstructured data retrieval agent) is generated by a retrieval agent module (e.g., retrieval agent module 506-1) and / or a structured data retriever agent module 506-3. In some embodiments, the second agent includes one or more third large language models (e.g., a large language model different from the first large language model and / or the second large language model).

[0180] In step 1010, the enterprise generative artificial intelligence system uses a second agent to retrieve one or more second data records from the structured data set. In some embodiments, the second agent uses (e.g., executes, accesses, and / or controls) one or more other tools (e.g., structured data retrieval tool module 508-2) to retrieve the structured data records. In some embodiments, the second agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from the structured data set (e.g., Figure 7 In some embodiments, the second agent instructs one or more structured data retrieval tools based on a data model that describes one or more types of relationships of the data model.

[0181] In step 1012, the enterprise generative artificial intelligence system utilizes an orchestrator to generate a natural language summary of one or more first data records and one or more second data records. In some embodiments, the orchestrator's large-scale language model generates the natural language summaries based on the natural language summaries generated by the first agent and the second agent. For example, the first agent may generate a natural language summary of a retrieved first data record, and the second agent may generate another natural language summary of a retrieved second data record.

[0182] At step 1014, the enterprise generative artificial intelligence system transmits a natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the input. In some embodiments, a communication module (e.g., communication module 532) transmits the natural language summary to the user and / or system that provided or generated the input.

[0183] In some embodiments, the orchestrator parses the input into different parts (e.g., segments) and routes each part to a corresponding agent. For example, the orchestrator may determine that a first segment requires an unstructured data retrieval operation and route the first segment to a first agent. Similarly, the orchestrator may determine that a second segment requires a structured data retrieval operation and route the second segment to a second agent. In some embodiments, the orchestrator may launch agents to process the input and / or orchestrate agents for parallel execution as needed.

[0184] Figure 11 Flowchart 1100 depicts an example generative artificial intelligence process using unstructured and structured data, according to some embodiments. This example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 1102, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) obtains a query. In some embodiments, an interface module (e.g., interface module 530) and / or an orchestrator module (e.g., orchestrator module 504) obtains the query (e.g., from a user and / or the system).

[0185] In step 1104, the enterprise generative artificial intelligence system interprets the query. Although a query is used in this example, it will be understood that other types of input may be used instead. In some embodiments, the orchestrator module interprets the query. In step 1106, the enterprise generative artificial intelligence system selects a first agent from a plurality of different agents based on the interpretation of the query. In some embodiments, the orchestrator module and / or another agent (e.g., unstructured data retriever agent module 506-1) selects the first agent (e.g., unstructured data retriever agent module 506-2).

[0186] In step 1108, the enterprise generative artificial intelligence system uses the first agent to retrieve one or more first data records from the unstructured data set. The first data record can be an unstructured data record retrieved from one or more vector data stores and / or based on one or more vector data stores. In some embodiments, the first agent uses (e.g., executes, accesses and / or controls) one or more tools (e.g., unstructured data retrieval tool module 508-1) to retrieve the unstructured data record. In some embodiments, the first agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process (e.g., Figure 8A and Figure 9 ) and / or non-iterative generative AI processes (e.g., as Figures 8B to 8C ) to retrieve one or more first data records from an unstructured dataset. In one example, the first agent instructs one or more unstructured data retrieval tools based on the embeddings in the vector store. For example, the first agent and / or tool can use an AI-based similarity search to identify and retrieve paragraphs with similar embedding values.

[0187] In step 1110, the enterprise generative artificial intelligence system selects a second agent from a plurality of different agents based on the interpretation of the query. In some embodiments, the orchestrator module and / or another agent (e.g., unstructured data retriever agent module 506-1) selects the second agent (e.g., unstructured data retriever agent module 506-2). In step 1112, the enterprise generative artificial intelligence system uses the second agent to retrieve one or more second data records from the structured data set. In some embodiments, the second agent uses (e.g., executes, accesses, and / or controls) one or more other tools (e.g., structured data retrieval tool module 508-2) to retrieve the structured data records. In some embodiments, the second agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from the structured data set (e.g., Figure 7 In some embodiments, the second agent instructs one or more structured data retrieval tools based on a data model that describes one or more types of relationships of the data model.

[0188] In step 1114, the enterprise generative artificial intelligence system generates a first natural language summary of one or more first data records. In some embodiments, the first agent generates the first natural language summary. In step 1116, the enterprise generative artificial intelligence system generates a visualization based on one or more second data records from the structured dataset. In some embodiments, a visualization agent module (e.g., visualization agent module 506-9) generates the visualization. For example, the visualization agent module may execute a visualization tool (e.g., visualization tool module 508-7) to generate the visualization. In step 1118, the enterprise generative artificial intelligence system generates a second natural language summary based on the first natural language summary and the visualization. In some embodiments, the orchestrator module generates the second natural language summary. In step 1120, the enterprise generative artificial intelligence system transmits the second natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the query. In some embodiments, a communication module (e.g., communication module 532) transmits the second natural language summary.

[0189] Figure 12 Flowchart 1200 depicts an example of a non-iterative generative artificial intelligence process using unstructured data, according to some embodiments. This example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 1202, the enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) receives input (e.g., complex input). In some embodiments, an orchestrator module (e.g., orchestrator module 504) receives the input. In step 1204, the enterprise generative artificial intelligence system retrieves a plurality of unstructured data records from a data store based on the input. In some embodiments, the orchestrator instructs an unstructured data retriever agent module (e.g., unstructured data retriever agent module 506-2) to retrieve the unstructured data records, and the agent instructs an unstructured data retrieval tool (e.g., unstructured data retrieval tool 508-1) to perform a retrieval function. In step 1206, the enterprise generative artificial intelligence system filters the plurality of unstructured data records. In some embodiments, the unstructured data retriever agent module instructs a filtering tool (eg, filter tool module 508 - 9 ) to filter the unstructured data records.

[0190] In step 1208, the enterprise generative artificial intelligence system identifies key points of the unstructured data records. In some embodiments, the unstructured data retriever agent module identifies the key points. In step 1210, the enterprise generative artificial intelligence system generates a summary of the filtered data records based on the identified key points. In some embodiments, an understanding module (e.g., understanding module 516) generates the summary. For example, the understanding module may include a large language model that takes the key points as input and generates a natural language summary based on the key points. As described elsewhere herein, the understanding module may be a component of the orchestrator.

[0191] In step 1212, the enterprise generative artificial intelligence system generates a result based on the summary. The result can be the summary itself or the output of another large language model. In some embodiments, the summary can be provided to another large language model (e.g., "answering" the large language model) to provide a contextualized final result. More specifically, the other large language model can generate results based on the user who submitted the initial query (e.g., based on the user role). In step 1214, the enterprise generative artificial intelligence system provides the result to the recipient associated with the input (e.g., the user or system that originally provided the input). In some embodiments, the communication module (e.g., communication module 532) provides the result via a communication network (e.g., communication network 408).

[0192] Figure 13 Flowchart 1300 depicts an example of a generative artificial intelligence process using structured data, according to some embodiments. This example process can be implemented by an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402). In step 1302, the enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) accesses a data model. The data model can include various types, and each type can describe different data fields, operations, and functions. In some embodiments, each type represents a different object (e.g., a machine in a factory), and each type can include context that enables another model (e.g., a large language model) to more accurately and / or efficiently process information and / or generate results. In some embodiments, an orchestrator (e.g., orchestrator module 504), an agent (e.g., agent 506), and / or a tool (e.g., tool module 508) accesses the data model. In certain implementations, a type system retriever agent (e.g., type system retriever agent module 506-4) accesses the data model.

[0193] In step 1304, the enterprise generative artificial intelligence system receives input (e.g., complex input). In some embodiments, the orchestrator module receives the input. In step 1306, the enterprise generative artificial intelligence system determines a plurality of keywords based on the input. In some embodiments, the orchestrator module determines the keywords.

[0194] In step 1308, the enterprise generative artificial intelligence system retrieves a subset of the data model based on a similarity search on the data model using multiple keywords. In some embodiments, a type system agent (e.g., type system retriever agent module 506-4) performs a similarity search and retrieves the subset of the data model. In step 1310, the enterprise generative artificial intelligence system retrieves structured data from one or more structured data stores based on the subset of the data model. In some embodiments, a structured data retriever agent (e.g., structured data retriever agent module 506-3) retrieves the structured data.

[0195] In step 1312, the enterprise generative artificial intelligence system determines, via the large language model, an output of the large language model based on the input and the corresponding large language model context of the type of the data model subset. In step 1314, the enterprise generative artificial intelligence system transmits the output of the large language model to a recipient associated with the input. In some embodiments, a communication module (e.g., communication module 532) transmits the output to the user and / or system that provided the initial input.

[0196] Figure 14 Flowchart 1400 depicts example operations of an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) according to some embodiments. In step 1402, the enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 402) obtains input (e.g., complex input). In some embodiments, an interface module (e.g., interface module 530) receives the input. In step 1404, the enterprise generative artificial intelligence system provides the input to an orchestrator (e.g., orchestrator module 504). The orchestrator may include one or more first large language models and / or one or more machine learning models. In some embodiments, the interface module provides the input to the orchestrator.

[0197] In step 1406, the enterprise generative artificial intelligence system utilizes the orchestrator to select inputs for the selected one or more agents (e.g., agent 506). In step 1408, the enterprise generative artificial intelligence system utilizes the orchestrator to route the inputs to the selected one or more agents.

[0198] In step 1410, the enterprise generative artificial intelligence system utilizes one or more corresponding tools associated with the selected one or more agents to generate corresponding outputs for each of the one or more tools. In some embodiments, the corresponding tools associated with the first agent include one or more retriever models, the corresponding tools associated with the second agent include a database query generator, the corresponding tools associated with the third agent include an API generator, and the corresponding retrieval tool associated with the fourth agent includes an iterative context-based generative artificial intelligence tool. The first portion of the input is routed to the first agent, the second portion of the input is routed to the second agent, and the third portion of the input is routed to the third agent. The first agent can be configured to generate one or more document retrieval requests, the second agent is configured to generate one or more SQL queries, the third agent is configured to generate one or more API calls for one or more artificial intelligence applications, and the fourth agent is configured to trigger the enterprise generative artificial intelligence system. The first agent, the second agent, and / or the third agent can execute in parallel with each other based on control instructions provided by the orchestrator.

[0199] In step 1412, the enterprise generative AI system transforms each of the one or more corresponding outputs into a natural language format. In some embodiments, the selected one or more agents transform the outputs. In step 1414, the enterprise generative AI system inputs the transformed outputs into a comprehension module. The comprehension module may include one or more third language models. The third large language model may be the same as or different from the first large language model. In step 1416, the enterprise generative AI system provides the outputs of the comprehension module to a recipient associated with the input.

[0200] Figure 15Flowchart 1500 depicts an example method for intelligent crawling and chunking according to some embodiments. In step 1502, a computing system (e.g., an enterprise generative artificial intelligence system and / or intelligent crawling and chunking subsystem 120) scans multiple different data domains of an enterprise information environment. In some embodiments, a crawling module (e.g., crawling module 514 and / or crawling module 122) scans the different data domains of the enterprise information environment. In step 1504, the computing system chunks multiple data records from multiple enterprise data sources across the multiple different data domains of the enterprise information environment. The data records may include any of documents, database tables, models, text, images, video, audio, artificial intelligence insights, application output, applications, source code, scripts, and / or compiled source code. Chunking may generate one or more data record segments for each of the multiple data records. In some embodiments, a chunking module (e.g., chunking module 510 and / or chunking module 124) chunks the data records.

[0201] In step 1506, the computing system generates corresponding contextual metadata for each of one or more corresponding data record segments. The contextual metadata may include contextual information and / or be generated based on contextual information. Each corresponding contextual metadata may indicate a semantic or contextual description of the corresponding data record segment. At least one of the corresponding contextual metadata may facilitate the determination of a relationship between one corresponding data record segment in a corresponding data record segment of a particular corresponding data record and another corresponding data segment in a corresponding data segment of another corresponding data record. The contextual description may indicate a particular data domain among a plurality of different data domains of an enterprise information environment. In some embodiments, the segmentation module generates the contextual metadata. The contextual metadata may indicate enterprise access control information and corresponding data domains, data records, and / or data record segments associated with an enterprise access control system of the enterprise information environment.

[0202] In step 1508, the computing system generates a corresponding segment embedding for each data record segment based on the corresponding segment and the corresponding contextual metadata. In some embodiments, an embedding generator module (e.g., embedding generator module 512) generates the segment embedding. The corresponding segment embedding can be a vector value. The corresponding segment embedding can include a corresponding vector embedding, and the embedding data store can include one or more vector data stores. In some embodiments, the contextual metadata is stored in a corresponding header of the corresponding segment and / or the corresponding data record.

[0203] In step 1510, the computing system stores the segment embeddings in an embedding data store. In some embodiments, the segment embeddings are stored in an embedding data store (e.g., embedding data store 540). For example, the embedding data store may include a vector data store, a model registry, and / or other data stores. In some embodiments, each segment comprises a corresponding sub-model of at least one of the models. In some embodiments, the models comprise different types of machine learning models. In some embodiments, the different types of machine learning models comprise a large language model. In some embodiments, scanning is performed continuously and / or on demand. In some embodiments, chunking is performed in parallel with scanning, and chunking is triggered based on one or more scanning operations of the scan. Scanning operations may include continuous scanning, on-demand scanning, identifying changes to data records, identifying new or deleted data records, and / or other operations described herein. In some embodiments, each data record segment is stored in a hierarchical structure in the embedding data store. For example, a data record may be a model, a data record segment may be a sub-model of a model, and the model and / or sub-model may be stored in one or more model registries and / or sub-models having a hierarchical structure.

[0204] In some embodiments, chunking and segmenting embedding facilitates one or more information retrieval processes. For example, the information retrieval process can be part of one or more generative artificial intelligence processes (e.g., generative artificial intelligence enterprise search).

[0205] In some embodiments, chunking is performed by an agent, segment embeddings are generated by another agent, and information is retrieved from an embedding store based on the corresponding embeddings by an additional agent, and wherein the agents are supervised by an orchestrator. For example, the additional agent may use a machine learning model to implement a similarity machine learning process that determines similarities between two or more segments and / or data records based on embeddings (e.g., vector embeddings generated based on data segments and contextual metadata). Thus, for example, similarity of segments may be determined based on similarity of the segments (e.g., similarity of paragraphs of the segments) and similarity of contextual metadata (or information indicated by the contextual metadata). In some embodiments, multiple data records may be processed by a type system before being scanned and chunked.

[0206] Figure 16 A diagram 1600 depicts an example data record pre-processing and information retrieval process according to some embodiments. The example data record pre-processing and information retrieval process can be performed by one or more of the systems and / or subsystems described herein (e.g., intelligent crawling and chunking subsystem 120 and / or enterprise generative artificial intelligence system 402).

[0207] Typically, data records may include information of different modalities, such as plain text, tables, images, and codes, etc. In order to efficiently and reliably retrieve information for a query, the preprocessing and information retrieval process provides a multimodal approach for extracting information from data records.

[0208] The pre-processing and information retrieval process may include three phases. The first phase may include parsing and extracting different modalities from the documents. This process may be performed in parallel (or substantially in parallel) for all different modalities.

[0209] In one example, text and code can be parsed (1602). Depending on the file format, extracting text information (i.e., plain text and code) can present different difficulties and may require the use of different libraries. Regardless of the data record type, the process can proceed through various steps to prepare for other downstream stages. Depending on the file format, the complexity of some of these steps may increase.

[0210] One of these steps may include extracting text information from data record 1601 (step 1604) so ​​that it can be further parsed. For example, extracting all content that is not an image or table. The output of this step may include (e.g., may need to include) all text information from the data record (e.g., information that is not an image or table title), which can then be used to further separate text and code segments. The parsed result can have high fidelity (e.g., no random spaces or strange characters that would destroy the meaning of a sentence) and be robust to the font, size, color, and position of other elements on the page.

[0211] Another step may include separating text and code (step 1605). The goal of this step is to identify and separate code and plain text. This then allows the system to further process these modalities separately. Additional steps may include chunking and parsing the text and code (step 1606). After separating the text and code modalities, the goal of this step is to identify, locate, and extract continuous code segments (step 1607) and chunk the text content in a reasonable and continuous manner (e.g., without mid-sentence or mid-paragraph breaks, particularly due to paging, and if possible, with chunks that are thematically continuous) (step 1606).

[0212] The system (e.g., intelligent crawling and chunking subsystem 120 and / or enterprise generative artificial intelligence system 402) can use an object-oriented structure, where there can be classes for plain text and code. These classes can have at least fields that track the content, the position in the document, and the number of tokens in the content (e.g., this means the system should know the tokenizer used for this purpose). For the text class, the system may already be able to track which code snippets have been removed from (or are associated with) the chunked content. This may already be done as part of the system.

[0213] In another example, tables can be parsed. An important modality in data records is tables. In order to enable efficient information retrieval, the system can first locate and identify the table in its entirety (e.g., because a table may span multiple pages / data records / segments or may appear in a single page with different structures). Step 1606. The system can identify libraries that make these features possible and measure their performance in fully identifying tables. Once the table is identified, it can be extracted as an image or data frame, etc. (step 1609). The system can also extract the title or name of the table, column headers, and possibly also (one or more than one) row indexes, etc. as metadata associated with it (step 1609). Similar to the text and code classes, the system can also have a table class that tracks the extracted table contents, its location, name / title, etc.

[0214] In another example, images can be parsed. Similar to how the system handles tables, the system can also begin by fully identifying and locating the image (step 1612). The system can include image classes (that is, content, location of the image in the document, and its title / name) to track images in the document as metadata associated with them. The system can also extract the image as metadata associated with it (step 1613).

[0215] At the end of the first phase, the system may have several instances of the text, code, table, and image classes, outlining the different modalities in each data record. After completing this, the system may proceed to the second phase, which is to build an information graph for each data record.

[0216] In the second phase, to facilitate efficient information retrieval, the system can use an information graph to represent the information in each data record 1601. Nodes 1630, 1641, 1643, and 1645 of this graph correspond to different instances of the four modalities from the first phase. The description is a (directed) bipartite graph with edges from text nodes to all other modal nodes. Establishing these edges is the primary goal of this phase.

[0217] exist Figure 16In the example of FIG, if there is a reference (e.g., a relationship) between a text node 1630 and other modal nodes 1641, 1643, and / or 1645, then there is an edge between them. This can be based on direct references to them in a text chunk, based on proximity, or even based on contextual similarity to their title or name. Once edges are identified, the system can track which edges are between each text node 1630 and other modal nodes 1641, 1643, and / or 1645 as part of a text class. Once the system has fully specified the graph, it can use the graph to design an information retrieval process.

[0218] In the third stage, given the information graph, the system can outline the process of information retrieval. One method for this starts by first embedding the content of the text node (and / or other text metadata associated with other modalities) (step 1629) and storing the embedding in the vector store 1626. Given a query 1622, the system can embed the query (step 1628) and find the most relevant text chunk or text node 1630 associated with it. This will be the entry point of the graph. At this point, the system can follow the outgoing edges to reach other modal nodes. Classes associated with these modalities (e.g., code, images, and tables) can have methods that enable the generation of relevant insights given the query. This method can be supported by different methods (including multimodal models or other tools for understanding and querying specific modalities). These insights, together with the text chunks and user queries, can then be combined in an aggregator 1650 into a text body or prompt for a query model (e.g., a multimodal model, a large language model, etc.).

[0219] In one implementation, Figure 16 The functionality shown and described in the foregoing can be performed by a chunking module (e.g., chunking module 510) and / or an embedding generator module (e.g., embedding generator module 512). For example, steps 1604-1613 can be performed by a chunking module, and steps 1628-1629 can be performed by an embedding generator module. In some embodiments, aggregator 1650 includes a portion of an orchestrator module (e.g., orchestrator module 504) and / or an understanding module (e.g., understanding module 516).

[0220] Figure 17Figure 1700 depicts an example of a computing device 1702. Any of the systems, engines, data stores, and / or networks described herein may include one or more instances of computing device 1702. In some embodiments, the functionality of computing device 1702 is modified to perform some or all of the functionality described herein. Computing device 1702 includes a processor 1704, memory 1706, storage 1708, input device 1710, a communication network interface 1712, and an output device 1714 communicatively coupled to a communication channel 1716. Processor 1704 is configured to execute executable instructions (e.g., a program). In some embodiments, processor 1704 includes circuitry or any processor capable of processing executable instructions.

[0221] Memory 1706 stores data. Some examples of memory 1706 include storage devices such as RAM, ROM, RAM cache, virtual memory, etc. In various embodiments, working data is stored in memory 1706. Data in memory 1706 can be cleared or eventually transferred to storage 1708.

[0222] Storage 1708 includes any storage configured to retrieve and store data. Some examples of storage 1708 include a flash drive, a hard drive, an optical drive, cloud storage, and / or magnetic tape. Each of memory system 1706 and storage system 1708 includes a computer-readable medium that stores instructions or programs executable by processor 1704.

[0223] Input device 1710 is any device for inputting data (e.g., a mouse and keyboard). Output device 1714 outputs data (e.g., a speaker or display). It will be understood that storage 1708, input device 1710, and output device 1714 may be optional. For example, a router / switch may include processor 1704 and memory 1706 as well as devices for receiving and outputting data (e.g., communication network interface 1712 and / or output device 1714).

[0224] The communication network interface 1712 can be coupled to a network (e.g., network 408) via a link 1718. The communication network interface 1712 can support communication via an Ethernet connection, a serial connection, a parallel connection, and / or an ATA connection. The communication network interface 1712 can also support wireless communications (e.g., 802.11 a / b / g / n, WiMax, LTE, Wi-Fi). It will be apparent that the communication network interface 1712 can support many wired and wireless standards.

[0225] It will be understood that the hardware elements of computing device 1702 are not limited to Figure 17Computing device 1702 may include more or fewer hardware, software, and / or firmware components than those depicted (e.g., drivers, operating system, touch screen, and / or biometric analyzer, etc.). Furthermore, hardware elements may share functionality and still be within the various embodiments described herein. In one example, encoding and / or decoding may be performed by processor 1704 and / or a coprocessor located on a GPU (i.e., NVidia).

[0226] Example types of computing devices and / or processing devices include one or more microprocessors, microcontrollers, reduced instruction set computers (RISCs), complex instruction set computers (CISCs), graphics processing units (GPUs), data processing units (DPUs), virtual processing units, associative processing units (APUs), tensor processing units (TPUs), vision processing units (VPUs), neuromorphic chips, AI chips, quantum processing units (QPUs), brain system wafer-level engines (WSEs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or discrete circuit systems.

[0227] It will be understood that an “engine,” “system,” “data store,” and / or “database” may include software, hardware, firmware, and / or circuitry. In one example, one or more software programs comprising instructions executable by a processor may perform one or more of the functions of the engine, data store, database, or system described herein. In another example, the circuitry may perform the same or similar functions. Alternative embodiments may include more, fewer, or functionally equivalent engines, systems, data stores, or databases and still be within the scope of the present embodiments. For example, the functionality of the various systems, engines, data stores, and / or databases may be combined or divided differently. The data store or database may include cloud storage. It will also be understood that the term “or,” as used herein, may be interpreted in an inclusive or exclusive sense. In addition, multiple instances may be provided for a resource, operation, or structure described herein as a single instance.

[0228] The data store described herein can be any suitable structure (e.g., an active database, a relational database, a self-referential database, a table, a matrix, an array, a flat file, a document-oriented storage system, and a non-relational No-SQL system, etc.), and can be cloud-based or otherwise. The systems, methods, engines, data stores, and / or databases described herein can be at least partially processor-implemented, wherein one or more specific processors are examples of hardware. For example, at least some of the operations of the method can be performed by one or more processors or processor-implemented engines. In addition, one or more processors can also operate to support the performance of related operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations can be performed by a computer group (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application program interfaces (APIs)).

[0229] The execution of certain operations may be distributed among processors, not only residing within a single machine, but also deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0230] Throughout the specification, multiple instances can be implemented as components, operations or structures described as single instances. Although the individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations can be performed simultaneously and do not need to be operated in the order shown. The structure and functionality presented as separate components in the example configuration can be implemented as combined structures or components. Similarly, the structure and functionality presented as separate components can be implemented as separate components. These and other variations, modifications, additions and improvements fall within the scope of this paper's theme.

[0231] The claimed solution, rooted in computer technology, overcomes problems particularly arising in the field of computer technology. Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to process input using an orchestrator. Selecting, using the orchestrator, a first agent from a plurality of different agents based on the processed input. Retrieving, using the first agent, one or more first data records from an unstructured data set. Selecting, using the orchestrator, a second agent from a plurality of different agents based on the processed input. Retrieving, using the second agent, one or more second data records from a structured data set. Generating, using the orchestrator, a natural language summary of the one or more first data records and the one or more second data records, and transmitting the natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the input.

[0232] In some embodiments, the orchestrator includes one or more first large-scale language models. In some embodiments, the first agent includes one or more second large-scale language models, and the second agent includes one or more third large-scale language models. The first agent uses one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve one or more first data records from an unstructured dataset. The second agent uses one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from a structured dataset. The first agent instructs the one or more unstructured data retrieval tools based on embeddings in the vector storage. Selecting, using the orchestrator, a first agent from among the plurality of different agents based on the processed input further includes generating a second input based on the first portion of the input and routing the second input to the first agent. Selecting, using the orchestrator, a second agent from among the plurality of different agents based on the processed input further includes generating a third input based on the second portion of the input and routing the third input to the second agent. The first agent and the second agent execute in parallel and perform their respective searches in parallel.

[0233] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: obtain a query; interpret the query; select a first agent from a plurality of different agents based on the interpretation of the query; retrieve one or more first data records from an unstructured dataset using the first agent; select a second agent from a plurality of different agents based on the interpretation of the query; retrieve one or more second data records from a structured dataset using the second agent; generate a first natural language summary of the one or more first data records; generate a visualization based on the one or more second data records from the structured dataset; generate a second natural language summary based on the first natural language summary and the visualization; and transmit the second natural language summary of the one or more first data records and the one or more second data records to a recipient associated with the query.

[0234] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: receive input; retrieve a plurality of unstructured data records from a data store based on the input; filter the plurality of unstructured data records; identify key points of the filtered data records; generate a summary of the filtered data records based on the identified key points; generate a result based on the summary; and provide the result to a recipient associated with the input.

[0235] In some embodiments, systems, methods, and non-transitory computer-readable media are configured to: determine a number of identified data records; compare the number to a threshold; and skip key point identification and generate a summary based on the filtered data records instead of the key points.

[0236] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: access a data model, wherein the data model includes a plurality of types, and wherein each type describes one or more corresponding data fields, operations, and functions, and wherein each type represents a different object, and wherein each type includes a corresponding large language model context; receive input; determine a plurality of keywords based on the input; retrieve a subset of the data model based on a similarity search on the data model using the plurality of keywords; retrieve structured data from one or more structured data stores based on the subset of the data model; determine, using the large language model, an output of the large language model based on the input and corresponding large language model contexts of types of the subset of the data model; and transmit the output of the large language model to a recipient associated with the input.

[0237] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to: obtain input; provide the input to an orchestrator, the orchestrator comprising one or more first large language models and one or more machine learning models; select, by the orchestrator, one or more agents from a plurality of agents, each agent comprising a corresponding second large language model and configured to access one or more corresponding tools from a plurality of different tools; route the input to the selected one or more agents using the orchestrator; generate corresponding outputs for each of the one or more tools using one or more corresponding tools associated with the selected one or more agents; transform each of the one or more corresponding outputs into a natural language format; input the transformed outputs into a comprehension module, the comprehension module comprising one or more third language models; and provide the outputs of the comprehension module to a recipient associated with the first input.

[0238] The orchestrator includes one or more second large language models. In some embodiments, the corresponding tool associated with the first agent includes one or more retriever models, the corresponding tool associated with the second agent includes a database query generator, the corresponding tool associated with the third agent includes an API generator, and the corresponding retrieval tool associated with the fourth agent includes an iterative context-based generative artificial intelligence tool. The first part of the input is routed to the first agent, the second part of the input is routed to the second agent, and the third part of the input is routed to the third agent. The first agent is configured to generate one or more document retrieval requests, the second agent is configured to generate one or more SQL queries, the third agent is configured to generate one or more API calls for one or more artificial intelligence applications, and the fourth agent is configured to trigger the enterprise generative artificial intelligence system. The first agent, the second agent, and the third agent execute in parallel with each other based on control instructions provided by the orchestrator. In some embodiments, the agents include one or more machine learning models. The above describes (one or more) present inventions with reference to example embodiments. It will be apparent to those skilled in the art that various modifications may be made and other embodiments may be used without departing from the broader scope of the present invention(s). Therefore, the present invention(s) is intended to cover these and other changes to the exemplary embodiments.

[0239] According to an example disclosed herein, a computer-implemented method is provided, comprising: utilizing an orchestrator to manage multiple agents to generate responses to inputs, wherein the orchestrator employs one or more large language models to process or deconstruct a prompt into a series of instructions for different agents, wherein each agent employs one or more machine learning models to process different inputs or different portions of the inputs associated with the prompts; utilizing the orchestrator to indicate a retrieval request related to the input to one or more of the multiple agents; receiving data from multiple data domains from one or more of the multiple agents based on instructions from the orchestrator; utilizing the orchestrator to analyze the retrieved data to formulate one or more responses to the prompts, wherein the orchestrator provides additional retrieval requests to the one or more agents to retrieve additional data that satisfies contextual validation criteria associated with the inputs; utilizing the orchestrator to output at least one of the following: a validated response from the one or more responses to the inputs that satisfies the contextual validation criteria; and a portion of the data retrieved by the one or more agents that is related to the inputs.

[0240] The additional retrieval request may include a request for additional data (which may be different from the retrieved data) that may be used to check, validate, and / or verify the retrieved data and / or one or more responses. For example, the additional retrieval request may include a request configured to retrieve data similar to the retrieved data but from one or more different data sources. The additional data may include data from a data domain (and / or a different portion of the same data domain) that is different from the data domain (and / or a corresponding portion of the data domain) from which the retrieved data was retrieved. Thus, the additional data may provide alternative data to the retrieved data based on which the retrieved data and / or one or more responses may be checked, validated, and / or verified.

[0241] Additional retrieval requests may be based on one or more responses to a prompt. For example, additional retrieval requests may be formulated to check or validate one or more responses to a prompt. Additional retrieval requests may, for example, include one or more queries formulated based on one or more responses to a prompt to query the content of the one or more responses to a prompt. The method may include indicating portions of data records and their corresponding data sources used to generate the retrieved data (and / or additional data) and / or the one or more responses to the prompt. Portions of data records and their corresponding data sources may be indicated as a portion of the retrieved data and / or a portion of the additional data. Additional retrieval requests may include requests to inspect data records (and / or their respective data sources) to check, validate, or verify one or more responses to a prompt.

[0242] The method may comprise validating one or more responses to the prompt. The validating may comprise comparing the one or more responses to the prompt with the additional data. For example, the method may comprise determining a measure of similarity and / or consistency between the one or more responses to the prompt and the additional data. The measure of similarity and / or consistency may be a numerical measure suitable for comparison with a threshold value. The validating may comprise validating the response to the prompt if the measure of similarity and / or consistency (between the response to the prompt and the additional data) is greater than a first threshold value. The validating may comprise not validating the response to the prompt if the measure of similarity and / or consistency (between the response to the prompt and the additional data) is not greater than a second threshold value (which may be the same as the first threshold value).

[0243] The request and receipt of additional data and / or verification of one or more responses to a prompt provides a synergistic technical effect of improving the integrity and reliability of the output provided in response to the prompt. These features are based on a technical understanding of the internal functionality of a model (e.g., a large language model) and / or the functions used to generate responses. For example, it has been recognized that such models and / or functions can be based on probabilistic methods that may, for example, generate some inconsistent or inaccurate results. Requesting and receiving additional data and / or verifying responses can avoid or mitigate the output of inconsistent or inaccurate results. Providing input-related portions of data retrieved by one or more agents in addition to providing verified responses can allow further confirmation of the results (e.g., by a user receiving the output), thereby further improving the integrity and accuracy of the results.

[0244] Processing or deconstructing the prompt into a series of instructions for different agents may include determining instructions for agents that depend on the nature of the agents. For example, the agents may include different agents for performing different functions (e.g., retrieving unstructured data, receiving structured data, etc.). The different agents may be configured to receive instructions in different formats and / or languages ​​(e.g., different programming languages). The instructions determined for the different agents may depend on the functions the agents are configured to perform and / or the formats and / or languages ​​the agents are configured to receive. For example, a first instruction may be determined for a first agent, wherein the first instruction is determined according to a format and / or language to be received and processed by the first agent (e.g., according to the format and / or language of the instructions that the first agent is configured to receive). A second instruction may be determined for a second agent, wherein the second instruction is determined according to a format and / or language to be received and processed by the second agent (e.g., according to the format and / or language of the instructions that the second agent is configured to receive). The format and / or language of the instructions determined for the first agent may be different from the format and / or language of the instructions determined for the second agent. One or more large language models employed by the orchestrator can be configured through training to determine instructions for an agent based on the format and / or language of instructions that the agent is configured to receive. That is, the large language model can be configured through training to output instructions in different formats and / or languages ​​for different agents. Model training can be implemented continuously, asynchronously, and using feedback (e.g., reinforcement learning, etc.). In some illustrative examples, the orchestrator can transform a first portion of the input into, for example, a SQL query and send it to an unstructured data retriever agent, and / or transform a second portion of the input into an API call and send it to an API agent.

[0245] At least one of the one or more agents can be configured to determine instructions for one or more tools to perform one or more operations (e.g., such as structured data retrieval and / or unstructured data retrieval, etc.). Different tools can be configured to receive instructions in different formats and / or languages ​​(e.g., different programming languages). The instructions determined for different tools can depend on the functions or operations that the tools are configured to perform and / or the formats and / or languages ​​that the tools are configured to receive. For example, a first instruction can be determined for a first tool (e.g., by a first agent), wherein the first instruction is determined based on the format and / or language to be received and processed by the first tool (e.g., based on the format and / or language of the instructions that the first tool is configured to receive). A second instruction can be determined for a second tool (e.g., by the first agent or by a second agent), wherein the second instruction is determined based on the format and / or language to be received and processed by the second tool (e.g., based on the format and / or language of the instructions that the second tool is configured to receive). The format and / or language of instructions determined for a first tool may be different than the format and / or language of instructions determined for a second tool. One or more machine learning models employed by one or more agents may be configured through training to determine instructions for a tool based on the format and / or language of instructions that the tool is configured to receive. That is, the machine learning model may be configured through training to output instructions in a given format and / or language for a given tool, and / or may be configured through training to output instructions in different formats and / or languages ​​for different tools.

[0246] Method can comprise: provide instruction and / or inquiry to code generation intelligent agent and / or tool.Code generation intelligent agent and / or tool can be configured to generate source code, machine code and / or other computer code.In some examples, code generation intelligent agent can be configured to instruct code generation tool to generate source code, machine code and / or other computer code.Code generation intelligent agent can be configured to determine what code is needed and / or should be generated for what intelligent agent, tool and / or other entity or application.Code generation intelligent agent can instruct code generation tool to generate the function that is suitable for receiving code and / or receive the language and / or format of intelligent agent, tool and / or other entity or application.Code generation intelligent agent and / or tool can output the code of the format and / or language that is configured to be received by specific intelligent agent, tool and / or other entity or application.

[0247] The method may include receiving data and converting the data into a natural language format to provide to a multimodal model (e.g., a large language model). For example, one or more agents may receive data and convert the data into a natural language format to provide to a multimodal model (e.g., a large language model) employed by an orchestrator. In some examples, one or more agents may receive data and summarize the content of the data in a natural language format to provide to a multimodal model (e.g., a large language model).

[0248] In various examples contemplated herein, an arranger, an agent and / or a tool can be configured to receive an input and generate an output based on the input. The output can be provided to another arranger, an agent and / or a tool for further processing. The output provided to another arranger, an agent and / or a tool can be configured in a format and / or language that can be read and processed by another arranger, an agent and / or a tool. An arranger, an agent and / or a tool (and / or a model adopted by an arranger, an agent and / or a tool) can be configured to determine the output of a format and / or language that can be read and processed by another arranger, an agent and / or a tool through training. By providing the output of a language and / or format to be read by another arranger, an agent and / or a tool, an arranger, an agent and / or a tool can collaborate with each other and interoperate to provide an overall output (e.g., in the form of at least one of the following: a verified response in one or more responses to an input that satisfies a contextual validation criterion; and a portion of the data retrieved by one or more agents that is relevant to the input). Using an orchestrator, one or more agents and / or one or more tools can allow different orchestrators, agents, and / or tools to perform different operations required to produce an overall output. This can allow different orchestrators, agents, and / or tools to be configured (e.g., through training) to perform different operations or functions. Different orchestrators, agents, and / or tools can perform different operations or functions in parallel. This can be used to improve the computational efficiency of providing outputs in response to inputs.

[0249] Outputting at least one of a validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input may include displaying a representation of at least one of the validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input on an electronic display. Outputting at least one of a validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input may include transmitting at least one of the validated response from one or more responses to an input that satisfies contextual validation criteria and a portion of data retrieved by one or more agents that is relevant to the input to a computing device for display at the computing device.

[0250] At least one of the validated responses in one or more responses to an input that satisfies contextual validation criteria and the portion of the data retrieved by the one or more agents that is related to the outputted input depends on the technical functionality of one or more multimodal models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more multimodal models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more multimodal models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more multimodal models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more multimodal models and / or the one or more machine learning models may include simulation of the one or more multimodal models and / or the one or more machine learning models using software.

[0251] The orchestrator may generate intermediate instructions to multiple agents, wherein the intermediate instructions include at least one of a portion of an input, a question related to the input generated by one or more multimodal models, and a follow-up question related to the answer generated by the one or more multimodal models. Outputting the portion of the data retrieved by the one or more agents related to the input may include a source reference to at least a portion of a verified response. Contextual validation criteria may include a threshold for identifying source material from an enterprise data system used to validate the response. Managing multiple agents may include iterative processing or multiple instructions from the orchestrator. Retrieving data from multiple data domains may include time series data, structured data, and unstructured data. At least one of the one or more agents may instantiate a tool to operate on the instructions, retrieved data, or intermediate data. The operation may include at least one of calculation, translation, formatting, and visualization. One or more agents may be trained on specific machine learning models for different domains. At least one agent may employ a type system to unify incompatible data from different data sources.

[0252] According to an example disclosed herein, a computer-implemented method is provided, comprising: utilizing an orchestrator to select one or more agents from a plurality of different agents based on processed input; utilizing the selected one or more agents to retrieve data records from an unstructured dataset and additional data records from a structured dataset; utilizing the orchestrator to generate a natural language summary of the data records from the unstructured dataset and the additional data records from the structured dataset; and transmitting the natural language summary of the data records and the additional data records as a response to the input.

[0253] The selected agent from a plurality of different agents may include a first agent configured to retrieve data records from an unstructured dataset and a second agent configured to retrieve additional data records from a structured dataset. The data records may be retrieved from the unstructured dataset by the first agent. The additional data records may be retrieved from the structured dataset by the second agent. The first agent may instruct one or more unstructured data retrieval tools to retrieve data records from the structured dataset. The first agent may be configured to generate instructions for the one or more unstructured data retrieval tools. The first agent may generate instructions in a format and / or language that the one or more unstructured data retrieval tools are configured to receive. The second agent may instruct one or more structured data retrieval tools to retrieve additional data records from the structured dataset. The second agent may be configured to generate instructions for the one or more structured data retrieval tools. The second agent may generate instructions in a format and / or language that the one or more structured data retrieval tools are configured to receive. The first agent and / or the second agent may employ (e.g., through training) one or more machine learning models configured to generate instructions for one or more data retrieval tools. The one or more machine learning models employed by the one or more agents may be configured through training to determine instructions for the tool based on the format and / or language of the instructions that the tool is configured to receive. That is, the machine learning model may be configured through training to output instructions in a given format and / or language for a given tool, and / or may be configured through training to output instructions in different formats and / or languages ​​for different tools. The machine learning model may include some or all of the different types or modalities of models described herein (e.g., multimodal machine learning models, large language models, data models, statistical models, audio models, visual models, audio-visual models, etc.).

[0254] The method may include receiving data and converting or packaging the data into a compatible format for provision to a multimodal model. For example, one or more agents may receive data and convert the data into a natural language format for provision to a large language model employed by an orchestrator. The data format may include some or all of the different types or modalities described herein (e.g., multimodal, textual, encoded, linguistic, statistical, audio, visual, audiovisual, etc.). In some examples, one or more agents may receive data and aggregate the content of the data in a natural language format for provision to a large language model.

[0255] Transmitting the natural language summary of the data record and the additional data record in response to the input may include transmitting the natural language summary of the data record and the additional data record to a computing device for display at the computing device. The transmitted natural language summary of the data record and the additional data record may depend on the technical functionality of one or more large language models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more large language models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more large language models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more large language models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more large language models and / or the one or more machine learning models may include emulation of the one or more large language models and / or the one or more machine learning models using software.

[0256] The orchestrator may include one or more large language models. The agents may include one or more large language models, and the additional agents may include one or more additional large language models. A first agent (of the one or more agents) may use one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve one or more data records from an unstructured dataset. A second agent (of the one or more agents) may use one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more additional data records from a structured dataset. The first agent may instruct the one or more unstructured data retrieval tools based on embeddings in a vector store. The second agent may instruct the one or more structured data retrieval tools based on a data model that describes one or more types of relationships in the data model. Selecting a first agent from a plurality of different agents based on the processed input using the orchestrator may also include generating an intermediate input based on a first portion of the input and routing the second input to the agent. Selecting, using the orchestrator, a second agent from the plurality of different agents based on the processed input may also include generating a third input based on the second portion of the input and routing the third input to the additional agent. The first agent and the second agent may execute in parallel and perform their respective searches in parallel.

[0257] According to an example disclosed herein, a system is provided that includes: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the system to: select, using an orchestrator, two or more agents from a plurality of different agents based on a received prompt; retrieve, using the two or more agents, one or more data records from an unstructured dataset and additional data records from a structured dataset; generate, using the orchestrator, a natural language summary of the one or more data records from the unstructured dataset and the additional data records from the structured dataset; and transmit, in response to the prompt, the natural language summary of the data records. The selected two or more agents from the plurality of different agents may include a first agent configured to retrieve data records from the unstructured dataset and a second agent configured to retrieve additional data records from the structured dataset. The data records may be retrieved from the unstructured dataset by the first agent, and the additional data records may be retrieved from the structured dataset by the second agent.

[0258] The first agent may instruct one or more unstructured data retrieval tools to retrieve one or more data records from a structured data set. The first agent may be configured to generate instructions for the one or more unstructured data retrieval tools. The first agent may generate instructions in a format and / or language that the one or more unstructured data retrieval tools are configured to receive. The second agent may instruct one or more structured data retrieval tools to retrieve one or more additional data records from a structured data set. The second agent may be configured to generate instructions for the one or more structured data retrieval tools. The second agent may generate instructions in a format and / or language that the one or more structured data retrieval tools are configured to receive. The first agent and / or the second agent may employ (e.g., through training) one or more machine learning models configured to generate instructions for the one or more data retrieval tools. The one or more machine learning models employed by the one or more agents may be configured to determine instructions for the tool based on the format and / or language of the instructions that the tool is configured to receive through training. That is, the machine learning model can be configured through training to output instructions in a given format and / or language for a given tool, and / or can be configured through training to output instructions in different formats and / or languages ​​for different tools.

[0259] The system may include a code generation agent and / or tool. The code generation agent and / or tool may be configured to generate source code, machine code and / or other computer code. In some examples, the code generation agent may be configured to instruct a code generation tool to generate source code, machine code and / or other computer code. The code generation agent may be configured to determine what code is needed and / or what agent, tool and / or other entity or application should generate code for. The code generation agent may instruct the code generation tool to generate code suitable for the function to be received and / or the language and / or format of the receiving agent, tool and / or other entity or application. The code generation agent and / or tool may output code in a format and / or language that is configured to be received by a specific agent, tool and / or other entity or application. Two or more agents may be configured to receive data and convert the data into a natural language format to provide to a large language model. For example, one or more agents may receive data and convert the data into a natural language format to provide to a large language model adopted by an composer. In some examples, one or more agents may receive data and summarize the content of the data in natural language format to provide to a large language model.

[0260] Transmitting the natural language summary of the data record in response to the prompt may include transmitting the natural language summary of the data record to a computing device for display at the computing device. The transmitted natural language summary of the data record may depend on the technical functionality of one or more large language models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more large language models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more large language models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more large language models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more large language models and / or the one or more machine learning models may include emulation of the one or more large language models and / or the one or more machine learning models using software.

[0261] The orchestrator may include one or more large language models. The first agent may employ one or more large language models, and the additional agent may employ a different large language model. The first agent may use one or more unstructured data retrieval tools to implement an iterative generative artificial intelligence process to retrieve one or more first data records from an unstructured dataset. The second agent may use one or more structured data retrieval tools to implement a non-iterative generative artificial intelligence process to retrieve one or more second data records from a structured dataset. The first agent may instruct the one or more unstructured data retrieval tools based on embeddings in a vector store. The second agent may instruct the one or more structured data retrieval tools based on a data model that describes one or more types of relationships of the data model.

[0262] Selecting, using the orchestrator, a first agent from among the plurality of different agents based on the processed input may further include generating a second input based on the first portion of the input and routing the second input to the first agent. Selecting, using the orchestrator, a second agent from among the plurality of different agents based on the processed input may further include generating a third input based on the second portion of the input and routing the third input to the second agent. The first agent and the second agent may execute in parallel and perform their respective searches in parallel.

[0263] According to an example disclosed herein, a non-transitory computer-readable medium is provided, comprising instructions that, when executed, cause one or more processors to: process an input using an orchestrator; select a first agent from a plurality of different agents based on the processed input using the orchestrator; retrieve one or more first data records from an unstructured data set using the first agent; select a second agent from a plurality of different agents based on the processed input using the orchestrator; retrieve one or more second data records from a structured data set using the second agent; generate a natural language summary of the one or more first data records and one or more second data records using the orchestrator; and transmit the natural language summary of the one or more first data records and one or more second data records to a recipient associated with the input.

[0264] The first agent may instruct one or more unstructured data retrieval tools to retrieve one or more data records from a structured data set. The first agent may be configured to generate instructions for the one or more unstructured data retrieval tools. The first agent may generate instructions in a format and / or language that the one or more unstructured data retrieval tools are configured to receive. The second agent may instruct one or more structured data retrieval tools to retrieve one or more additional data records from a structured data set. The second agent may be configured to generate instructions for the one or more structured data retrieval tools. The second agent may generate instructions in a format and / or language that the one or more structured data retrieval tools are configured to receive. The first agent and / or the second agent may employ (e.g., through training) one or more machine learning models configured to generate instructions for the one or more data retrieval tools. The one or more machine learning models employed by the one or more agents may be configured to determine instructions for the tool based on the format and / or language of the instructions that the tool is configured to receive through training. That is, the machine learning model can be configured through training to output instructions in a given format and / or language for a given tool, and / or can be configured through training to output instructions in different formats and / or languages ​​for different tools.

[0265] Code generation agents and / or tools can be used to generate source code, machine code and / or other computer code. In some examples, the code generation agent can be configured to instruct the code generation tool to generate source code, machine code and / or other computer code. The code generation agent can be configured to determine what code is needed and / or for which agent, tool and / or other entity or application to generate code. The code generation agent can instruct the code generation tool to generate code suitable for the function to be received and / or the language and / or format of the receiving agent, tool and / or other entity or application. The code generation agent and / or tool can be configured to output code in the format and / or language received by a specific agent, tool and / or other entity or application. For example, two or more agents can be configured to receive data and convert the data into a natural language format to provide to a large language model. For example, one or more agents can receive data and convert the data into a natural language format to provide to a large language model adopted by an assembler. In some examples, one or more agents can receive data and summarize the content of the data in a natural language format to provide to a large language model.

[0266] Transmitting a multimodal summary (e.g., a natural language summary) of the data record in response to a prompt may include transmitting the natural language summary of the data record to a computing device for display at the computing device. The transmitted natural language summary of the data record may depend on the technical functionality of one or more multimodal models (e.g., a large language model employed by the orchestrator) and / or one or more machine learning models (e.g., a machine learning model employed by the agent). The one or more large language models and / or the one or more machine learning models may have been trained using machine learning. Thus, the operation of the one or more large language models and / or the one or more machine learning models may be based on parameters that have been learned through training (as opposed to parameters that have been set by a human programmer). The one or more large language models and / or the one or more machine learning models may be implemented in dedicated hardware. Additionally or alternatively, the one or more large language models and / or the one or more machine learning models may include emulation of the one or more large language models and / or the one or more machine learning models using software.

Claims

1. A method comprising: Scan multiple different data domains of the enterprise information environment; Chunking a plurality of data records from a plurality of enterprise data sources in a plurality of different data domains of the enterprise information environment, the chunking generating one or more corresponding data record segments for each data record in the plurality of data records; generating respective contextual metadata for each respective data record segment of the one or more respective data record segments, each respective contextual metadata indicating a semantic or contextual description of the respective data record segment, and at least one of the respective contextual metadata being capable of facilitating determining a relationship between one of the respective data record segments of a particular respective data record and another of the respective data segments of another respective data record; generating a corresponding segment embedding for each data record segment based on the corresponding context metadata; as well as The segment embedding is stored in an embedding data store.

2. The method according to claim 1, wherein The contextual metadata indicates a specific data domain among a plurality of different data domains of the enterprise information environment.

3. The method according to claim 1, wherein The data records include any of documents, database tables, models, text, images, video, audio, artificial intelligence insights, application output, applications, source code, scripts, and compiled source code.

4. The method according to claim 1, wherein The context metadata indicates any of corresponding data domains, data records, and data record segments, and enterprise access control information associated with an enterprise access control system of the enterprise information environment.

5. The method according to claim 1, wherein The respective segment embeddings include vectors, and the embedding data store includes one or more vector data stores.

6. The method according to claim 1, wherein The context metadata is stored in the corresponding header of the corresponding segment.

7. The method according to claim 3, wherein: Each segment includes a corresponding sub-model of at least one of the models.

8. The method according to claim 7, wherein: The models include different types of machine learning models.

9. The method according to claim 8, wherein The different types of machine learning models include large language models.

10. The method according to claim 1, wherein The scanning is performed continuously and / or on demand.

11. The method according to claim 1, wherein The blocking is performed in parallel with the scanning, and the blocking is triggered based on one or more scanning operations of the scanning.

12. The method according to claim 1, wherein Each of the data record segments is stored in a hierarchical structure in the embedded data store.

13. The method according to claim 12, wherein: One or more information retrieval processes include a portion of one or more generative artificial intelligence processes.

14. The method according to claim 1, wherein The chunking is performed by an agent, the segment embeddings are generated by another agent, and information is retrieved from an embedding store by the additional agent based on the corresponding embeddings, and wherein the agents are supervised by an orchestrator.

15. The method according to claim 1, wherein The plurality of data records are processed by a type system before being any of scanned and chunked.

Images