Generative artificial intelligence enterprise search

Abstract

Systems and methods are configured to utilize data from at least a plurality of data domains of an enterprise information environment including access control to generate a set of potential responses to hints using one or more data models. A deterministic response is selected from the set of potential responses based on the scoring of the verification data and taking into account the configuration file information associated with the hint is restricted based on the access control. These enterprise generative AI systems and methods support granular enterprise access control, privacy and security requirements. The enterprise generative AI provides traceable references and links to insight underlying source information into the generative AI. These systems and methods enable significant increase in utility of enterprise users on information, analyses, and predictive analyses associated with and derived from a combination of enterprise and external information systems.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence and machine learning. More specifically, the present disclosure relates to a generative artificial intelligence-based system for transforming information access and content creation for enterprise information systems. Background Art

[0002] Conventional approaches typically require users to possess technical expertise commensurate with the complexity designed into the computer system and / or the difficulty of the task the user is directing the computer system to perform. Users may need to be familiar with the computer system's capabilities, resources, and / or data availability. This reduces human-computer interaction. Advanced computing systems typically require user training and sophisticated users to understand the complexity of the system's capabilities. Furthermore, enterprise systems with complex interfaces increase the barriers for users to retrieve useful information and hinder access to complex applications. Consequently, enterprise systems and users cannot effectively or efficiently utilize enterprise data and applications. Summary of the Invention

[0003] Disclosed herein are novel systems and methods for applying generative artificial intelligence to transform information access and content creation for enterprise systems. These enterprise generative AI systems and methods fundamentally change the human-computer interaction (HCI) model for enterprise software. These enterprise generative AI systems and methods combine user search with the capabilities of natural language processing, generative AI, and predictive analytics to enable enterprise users to ask open-ended, multi-level, context-specific questions. Enterprise generative AI can receive natural language input and process it using machine learning to understand the request, identify and score relevant information, and authorize information interaction in enterprise environments involving large amounts of data and complex enterprise software.

[0004] These enterprise generative AI systems and methods provide maximum compatibility and adaptability across a variety of deployment factors. Enterprise generative AI is deployable across different data domains (e.g., different data sources, classified data, industry-specific data, etc.), with external interfaces and APIs that can operate in both connected and non-connected computing environments. Enterprises running sensitive workloads in cloud-native, on-premises, or air-gapped environments can implement enterprise generative AI to enable enterprise-wide collaboration and knowledge sharing, with centralized, automated, and continuous indexing of key knowledge across the entire corpus of the enterprise's information systems, or a subset thereof. These enterprise generative AI systems and methods support granular enterprise access control, privacy, and security requirements. Enterprise generative AI as disclosed herein provides traceable references and links to the source information underlying the generative AI insights.

[0005] This innovation enables enterprise users to significantly increase their access to information, analysis, and predictive analytics associated with and derived from a combination of enterprise and external information systems. Aspects of this innovation enable enterprise and related external information to be easily accessed by a wider range of users within the enterprise (such as untrained users, domain experts, senior managers, front-line workers, customers, etc.). An intuitive natural language interface provides functional access to application-specific interfaces, operations, and retrieval processes to automate and reduce the steps performed by users. Enterprise users can interact with enterprise generative AI through a variety of output formats and interfaces, including: automatic summary charts of key information, AI-generated summary answers of reference sources, email briefings, alerts, general content generation (e.g., proposals), generative AI-driven chat interfaces, ranked lists of top results, etc. HCI can include a simplified natural language interface and advanced accessibility features for adaptable forms of input, where adaptable forms of input include but are not limited to voice, text, and menu controls.

[0006] The claimed solution, rooted in computer technology, overcomes problems that arise particularly in the field of computer technology. Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to receive a query. In an example implementation, multiple enterprise data sets, artificial intelligence applications, and data models from different data domains of an enterprise information environment are identified based on the query. A set of relevance scores is generated for different portions of the enterprise data set determined based on data models from different data domains. Response information from the different data domains is determined by a machine learning model (such as a multimodal or large language model) based on the relevance scores and access control protocols. Natural language output is generated based on the response information from the relevant data domains. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A diagram depicting an example enterprise generative artificial intelligence system architecture and process, according to some embodiments.

[0008] Figure 2A Depicted is an example enterprise generative artificial intelligence query graphical user interface and architecture, according to some embodiments.

[0009] Figure 2B Depicted is an example enterprise generative artificial intelligence response graphical user interface in accordance with some embodiments.

[0010] Figure 3 A diagram depicts an example network system for enterprise generative artificial intelligence, according to some embodiments.

[0011] Figure 4 A diagram depicting an example of enterprise generative artificial intelligence, according to some embodiments.

[0012] Figure 5 Depicted is a flow diagram of an example enterprise generative artificial intelligence method, according to some embodiments.

[0013] Figure 6 Depicted is a flow diagram of an example enterprise generative artificial intelligence method, according to some embodiments.

[0014] Figure 7A and Figure 7B A diagram depicts an example enterprise generative artificial intelligence response graphical user interface, in accordance with some embodiments.

[0015] Figure 8 A diagram depicting an example system architecture and flow for an iterative enterprise generative artificial intelligence process, according to some embodiments.

[0016] Figure 9 Depicted is a flow diagram of an example enterprise generative artificial intelligence method, according to some embodiments.

[0017] Figure 10 Depicted is a flow diagram of an example enterprise generative artificial intelligence method, according to some embodiments.

[0018] Figure 11 is a diagram of an example computer system for implementing features disclosed herein, according to some embodiments. DETAILED DESCRIPTION

[0019] Enterprise environments typically have dozens, if not hundreds, of different applications with user interfaces specifically designed for accessing enterprise information. Each enterprise application may have a series of user interfaces that a user must navigate to access operations and information. Some enterprise applications even have different user interfaces for different roles and operations, thereby multiplying the potential permutations of steps a user must learn and perform to access enterprise information. Application user interfaces are typically designed to strike a balance between making operation of the application possible and the learning curve required for user adoption. Although user interfaces can be designed to highlight frequent operations and reduce navigation to frequent operations, the full utility of such enterprise applications becomes buried and complex to access without familiarity and / or training. In addition, enterprise environments have ever-increasing amounts of data. Some AI enterprise platforms and applications have successfully ingested large amounts of diverse enterprise information by utilizing AI-based methods for predictive analytics, such as anomaly detection, natural language processing (NLP), object detection, and forecasting. For example, the C3 AI platform with C3 AI applications (such as C3 AI Reliability, C3 AI Supply Chain, C3 AI Sustainability, C3 AI CRM, C3 AI ERP, C3 AI Defense, C3 AI Energy, etc.) provides flexibility in fine-tuning decision making. As new enterprise applications are inevitably added to the enterprise environment, the user knowledge requirements for effectively using multiple legacy and emerging technologies to access information can become overwhelming.

[0020] In conventional computing systems, users are typically required to have technical expertise commensurate with the complexity designed into the computer system and / or the difficulty of the task the user is instructing the computer system to perform. Users may need to be familiar with the capabilities, resources, and / or data availability of the computer system. This reduces human-computer interaction. Advanced computing systems typically require user training and sophisticated users to understand the complexity of the system's capabilities. In addition, enterprise systems with complex interfaces increase the barriers for users to retrieve useful information and prevent access to complex applications. For example, predictive analytics software applications typically require domain experts to execute their complex capabilities. Interaction with complex computer systems requires intuitive interfaces that enable a variety of users to effectively and quickly execute complex software functions while maintaining access, privacy, and security controls.

[0021] Aspects of the present disclosure relate to a framework for unifying information access methods and application operations across legacy and new enterprise applications and the growing size of data sources in enterprise environments. Systems and methods are described for coordinating access to information and increasing the availability of complex application operations while complying with enterprise security and privacy controls. The framework described herein uses machine learning techniques to navigate enterprise information and applications, understand organization-specific contextual cohorts (e.g., acronyms, nicknames, jargon, etc.), and locate the information most relevant to a request. Example aspects of the present disclosure lower the learning curve and reduce the steps a user must take to access information, thereby democratizing the use of information currently prevented by the complexity and domain expertise required by conventional enterprise information systems.

[0022] Enterprise information systems (or simply "enterprise systems") present numerous technical challenges that make them incompatible with conventional generative AI solutions, both from a technical and user experience perspective. For example, enterprise systems can store sensitive data, encompass a variety of disparate data sources and applications, operate in cloud-native, on-premises, and air-gapped environments, and be accessed by diverse user types with varying levels of enterprise permissions and varying amounts of technical knowledge. Furthermore, traditional enterprise information retrieval is slow, inefficient, unintuitive, unreliable, and inaccurate. Conventional generative AI solutions do not suffer from any of the problems posed by enterprise systems. In fact, they typically exacerbate these problems. For example, conventional generative AI solutions often provide unreliable or random responses, which are presented as both reliable and accurate without any traceability or accountability. Conventional generative AI solutions are also prone to hallucinations and information leakage (e.g., because conventional generative AI solutions cannot effectively separate enterprise data from generative AI models), and they cannot effectively leverage information across different data domains. Conventional generative AI solutions also rely on specific generative AI models that cannot be adapted to address enterprise-specific requirements or changes in the enterprise environment. These and other issues are addressed by the enterprise generative AI systems, methods, architectures, and features discussed in this paper.

[0023] These systems, methods, architectures, and features enable intuitive, non-complex interfaces to quickly execute complex user requests with improved access, privacy, and security enforcement. Generative AI techniques are leveraged to enable responses to user requests to be provided in an intuitive, non-complex manner. An enterprise understanding module is used to understand the language, intent, and context of a user's natural language query to discern relevant information from the enterprise information environment and generate deterministic responses.

[0024] Example aspects include systems and methods for implementing machine learning models (such as multimodal models, large language models (LLMs), and other machine learning models with enterprise-level integrity (including access control, traceability, anti-hallucination, and data leakage protection). The machine learning models may include some or all of the different types or modalities of models described herein (e.g., multimodal machine learning models, large language models, data models, statistical models, audio models, visual models, audio-visual models, etc.). Traceability functions enable the ability to trace back to source documents and data for each generated insight. Data protection elements protect data (e.g., confidential information) from being leaked or contaminating inherited model knowledge. Generative AI systems provide various features specifically designed to address the requirements and challenges posed by enterprise systems and environments. Enterprise generative AI systems can securely, efficiently, and accurately use generative AI methods, algorithms, and multimodal models (e.g., large language models and other machine learning models) to provide deterministic responses (e.g., in response to natural language queries and / or other instruction sets) that leverage enterprise data across different data domains, data sources, and applications. Data can be stored and / or accessed separately and differently from the generative AI model. The enterprise generative AI system prevents the use of enterprise data or portions thereof (e.g., sensitive enterprise data) to train the generative AI system's large language model. This provides deterministic responses without hallucinations or information leakage. The framework is adaptable and compatible with different large language models, machine learning algorithms, and tools.

[0025] Access controls enforce restrictions imposed by administrative and / or security policies, profile permissions, organizational controls, and so on. Access controls can be implemented at various stages of generative AI to prevent ingestion of models, prompts, results, and so on. For example, restricted data can be permitted to be ingested by an LLM, but the results can be omitted, suppressed, masked, or abstracted according to the access control policy. Enterprise generative AI systems can use role-based access controls that can prevent users from receiving responses from generative AI systems that include information that is inappropriate for their enterprise permissions.

[0026] The generative AI model interacts with one or more retrieval multimodal models. The one or more retrieval models are used to understand the underlying data, documents, and applications of the enterprise information environment. The underlying data of the enterprise information environment can be embedded by an orchestration module, for example, by a model-driven architecture for conceptual representation of enterprise and external data sets for data virtualization and access control. The information presented using predictive analytics can include text summaries, ranked results, smart cards, AI-driven chat interfaces, instantaneous generated content, etc. The predictive analytics can come from one or more AI applications including, for example, supply chain, CRM, ERP, reliability, defense, sustainability, or energy. Generative AI systems can be deployed across cloud-native, on-premises, and air-gapped environments, and the generative AI models used by the systems can be easily and efficiently changed (or "swapped"). Therefore, once the system has been deployed in an enterprise environment, these multimodal generative AI systems are not restricted or locked into a specific type, format, or large language model. Multimodal generative AI systems increase the compatibility and accessibility of data across different data domains.

[0027] In addition, the enterprise generative AI system can also obtain and / or trigger AI insights generated by enterprise AI applications when determining responses. For example, in response to receiving a query (e.g., via an intuitive graphical user interface), the system can trigger one or more AI applications to generate insights (e.g., based on enterprise data), and then provide a deterministic response based on generative AI, which includes a natural language summary of the insight and a traceability report indicating the source information used to determine the response. Thus, in this architecture, users and systems can safely and reliably utilize enterprise data and applications while also being able to trust the response.

[0028] Figure 1 A diagram depicts an example enterprise generative artificial intelligence system architecture and environment 100, according to some embodiments. Figure 1In an example of , the system architecture and environment 100 includes an enterprise generative artificial intelligence system 102, an enterprise system 104, an external system 106, a domain model 108, a vector data store 126, an embedded model data store 124, and an enterprise access control layer 115. In some embodiments, the query includes a natural language query received through a graphical user interface. In some embodiments, one or more enterprise datasets include any of documents, document fragments, and insights generated by one or more artificial intelligence applications. In some embodiments, each relevance score is associated with a corresponding portion of the one or more enterprise datasets, wherein each relevance score is determined relative to other corresponding portions of the one or more enterprise datasets. In some embodiments, the systems, methods, and non-transitory computer-readable media are further configured to generate a traceability analysis of the natural language output, the traceability analysis indicating any of the documents, document fragments, and insights of the corresponding portion of the one or more enterprise datasets.

[0029] Each data model in the plurality of data models may correspond to a different data domain in the plurality of different data domains. In some embodiments, each data model represents corresponding relationships and attributes corresponding to the different data domain in the plurality of different data domains. The corresponding relationships and attributes include any of data types, data formats, and industry-specific information. In some embodiments, the natural language output includes a summary of at least one of the corresponding portions of the one or more enterprise datasets associated with a relevance score.

[0030] In general, the enterprise generative AI system 102 can be used to securely query and process enterprise data and applications across different domains of an enterprise information environment. This can be referred to as generative enterprise search (or simply enterprise search). As shown, the enterprise generative AI system 102 can receive a question 116 (e.g., input, prompt, natural language query, instruction, etc.). In general, the enterprise generative AI system 102 can use a large language model 120 and a retrieval model 122 to process the query. More specifically, the enterprise generative AI system 102 can use the large language model 120 to interpret, understand, and / or resolve the query. The retrieval model 122 can interact with the large language model 120 and the domain model 108 to retrieve data records (e.g., documents, images, application output, AI insights and objects, etc.) across different domains using domain-specific data models 112. Therefore, the enterprise generative AI system 102 can use the large language model 120, the retrieval model 122, and the domain model 108 to generate accurate, reliable, and secure enterprise search results.

[0031] The enterprise generative artificial intelligence system 102 can use connectors 110, data models 112, and various persistence mechanisms and technologies 114 to facilitate the ingestion and persistence of enterprise system data from the enterprise system 104 and / or external system data from external systems 106 (e.g., systems outside the enterprise information environment). The enterprise system 104 may include a CRM system, an EAM system, and / or an ERP system, among others, and the connectors can facilitate the ingestion of data from various data sources (e.g., an Oracle system and / or an SAP system, among others). In some embodiments, the data model 112 provides attributes, relationships, and / or functionality associated with a particular domain. For example, the domains may include the aerospace domain 112-1, the energy domain 112-2, and / or the defense domain 112-3, among others. The domain model 108 can enable the enterprise generative artificial intelligence system 102 to provide domain-specific results without regard to the security or integrity of the underlying enterprise data, systems, and applications.

[0032] Furthermore, the enterprise generative AI system 102 utilizes or manages an enterprise access control layer 115, which can provide numerous technical benefits. In some embodiments, the enterprise access control layer 115 facilitates the separation of underlying enterprise information (e.g., enterprise data, applications, systems) from the large language model 120 and / or other machine learning models of the enterprise generative AI system 102. Thus, the enterprise generative AI system 102 can provide domain-specific, deterministic results without having to train the large language model 120 and / or other machine learning models on such enterprise information, which can lead to the myriad problems discussed above (e.g., information leakage, hallucinations).

[0033] In some embodiments, the enterprise generative artificial intelligence system 102 may use an enterprise access control layer 115 to implement additional enterprise controls. For example, an enterprise information environment may include users and systems with different enterprise permission levels. The enterprise access control layer 115 can ensure that responses or outputs comply with access and security protocols. The enterprise generative artificial intelligence system 102 protects information so that users are permitted to access data based on permissions, profiles, and controls. In one example, the enterprise access control layer 115 can filter information restricted by the retrieval model 122 before the large language model 120 processes or presents an answer or other output. More specifically, the enterprise access control layer 115 can filter data sources, data records, and / or other elements of the enterprise information environment so that query responses (or supporting traceability references) do not include information that the user is not permitted to access.

[0034] The enterprise generative AI system can also perform similar functionality based on the context of the user and / or system submitting the query. For example, a director and an engineer can submit the same query (e.g., "What projects are overdue?"), and the enterprise generative AI system 102 can use contextual information (e.g., user role, permissions, and domains associated with the user, etc.) to provide a contextual response in terms of its nature (e.g., providing information about overdue projects for a specific requester) and / or in terms of the presentation of the response (e.g., an engineer can receive more detailed technical information, while a director can receive less technical details).

[0035] In some embodiments, the enterprise generative artificial intelligence system 102 can use contextual information (e.g., contextual metadata) and data record embeddings to crawl, index, and / or map a corpus of data records (e.g., data records of one or more enterprise systems or environments) to provide access control (e.g., role-based access), provide improved data record identification and retrieval, and map relationships between data records. In one example, contextual information can prevent some users from accessing (e.g., viewing, retrieving) certain data records and improve similarity evaluations used in retrieval operations (e.g., processed by the generative artificial intelligence).

[0036] In some implementations, the enterprise generative artificial intelligence system 102 can generate embeddings based on the embedding model of the embedding model data store 124 and the content of the data record. In some implementations, the embedding can be represented by one or more vectors that can be stored in the vector data store 126. In some implementations, the retrieval model 122 can use the embedding to retrieve related data records and perform similarity or relevance evaluations or other aspects of the retrieval operation. As used herein, data records can include unstructured data records (e.g., documents and text data stored on a file system in formats such as PDF, DOCX, .MD, HTML, TXT, PPTX, image files, audio files, video files, and application output), structured data records (e.g., database tables or other data records stored according to a data model or type system), time series data records (e.g., sensor data, artificial intelligence application insights), and / or other types of data records (e.g., access control lists).

[0037] Figure 2A An example enterprise search graphical user interface 200 and underlying architecture 202 through 208 are depicted in accordance with some embodiments. Figure 2A and Figure 2B The graphical user interface depicted in includes a human-machine interface for receiving natural language queries and presenting relevant information from an enterprise information environment in response to the queries. Figure 2A or Figure 2B Not shown, but relevant information may also include visualizations, predictive analytics, and / or other information obtained or generated by the enterprise generative AI system.

[0038] exist Figure 2A In an example of , the diagram includes an enterprise query input interface 200 and frameworks 202 to 208 for unifying information access methods and application operations across legacy and new enterprise applications and the growing size of data sources in various enterprise environments. The enterprise generative artificial intelligence system described herein can coordinate access to information and increase availability for complex application operations while complying with enterprise security and privacy controls. The framework uses machine learning techniques (e.g., generative artificial intelligence algorithms and models) to navigate enterprise information and applications, understand organization-specific contextual cohorts (e.g., acronyms, nicknames, jargon, etc.), and locate the information most relevant to a request (e.g., a query). This can, for example, lower the learning curve and reduce the steps a user must take to access information, thereby democratizing the use of information that is currently prevented by the complexity and domain expertise required by conventional enterprise information systems.

[0039] exist Figure 2A In the example of , the enterprise query input interface 200 includes graphical user interface elements configured to receive various inputs (such as natural language queries, etc.). For example, a user can ask the system "Why did my brakes fail?", and the enterprise generative artificial intelligence system can utilize a natural language processing component 202 and one or more generative pre-trained transformers 204 (e.g., a large language model and / or other machine learning models) to process the query to generate answers safely, accurately, and reliably based on a variety of different enterprise data stores and applications 208. More specifically, the natural language processing component 202 and the generative pre-trained transformers 204 can generate one or more new queries 206 to process the initial user query. For example, a first new query can include an SQL query to retrieve data records from a relational database management system, and a second query can include another type of query (e.g., an instruction set) to execute one or more applications so that the results of the application execution can be returned and used by the system to generate an enterprise search response. The example enterprise search response interface is in Figure 2B shown in and described below.

[0040] In some embodiments, Figure 2A and Figure 2B The enterprise search interface shown and described in can be generated by the enterprise generative artificial intelligence system described herein, and frameworks 202 to 208 can represent the enterprise generative artificial intelligence system architecture and environment described herein.

[0041] Figure 2BAn example enterprise generative artificial intelligence response graphical user interface 250 is depicted in accordance with some embodiments. In some embodiments, the enterprise generative artificial intelligence response graphical user interface 250 may be generated, at least in part, by the enterprise generative artificial intelligence system described herein. Figure 2B In the example of , the enterprise generative artificial intelligence response graphical user interface 250 includes an enterprise search query input portion 252 , a generative enterprise search results portion 254 , and an interactive query portion 256 .

[0042] The enterprise search query input portion 252 presents an enterprise search query 258. In some implementations, the query can be entered through the input portion 252, although it may also have been entered in another interface (e.g., Figure 2A ) and is presented in input portion 252 as part of a generative enterprise search response.

[0043] Generate enterprise search result portion 254 includes a generative artificial intelligence response type 260, a generative artificial intelligence response condition 262, a generative artificial intelligence enterprise search result 266, a source data portion for generating a response 268, a source identifier 269, and a generative artificial intelligence response feedback element 270. Figure 2B In the example of , response type 260 is a summary, but there may be other response types that the enterprise generative artificial intelligence system can generate. The response status indicates the status of the response. The response status can include, for example, the processed query, the processed query, the evaluated metrics, the searched documents, the generated answers (e.g., results), the generated visualizations (e.g., time series visualizations for presentation in the response graphical user interface), and / or the completed generation (e.g., as shown in FIG. Figure 2B shown).

[0044] Source data portion 268 includes at least a portion of the information from the source data used to generate the response. This can, for example, enable a user to trust the response without having to independently verify the response. Source identification 269 identifies the source data record used to generate the response. For example, the source identification can indicate the entity name, field type, description or name of the data record (e.g., service manual, user manual, and / or technical manual, etc.), and / or the type of data record (e.g., document, or more specifically, PDF document), etc. This can also provide traceability and enable a user to trust the response.

[0045] Response feedback section 270 enables the user to provide feedback (e.g., positive or negative feedback) about the response. The enterprise generative artificial intelligence system can use the received feedback to improve the enterprise generative artificial intelligence system (e.g., through reinforcement learning), for example.

[0046] Interactive query section 256 enables the user to enter additional related queries (e.g., "follow-up" questions) via interactive input section 257. Figure 2B In the example of FIG, the interactive query portion 256 includes a chat interface, but other interfaces may use different interactive query portions. The interactive query portion 256 also includes a system-generated message 274 that prompts the user to ask follow-up questions, and the user may provide additional related queries 276. The interactive query portion 256 may also include a status portion 278 that indicates the status of the processing of the additional related queries 276. The status may include, for example, the processing of the query (e.g., Figure 2B as shown), processed queries, evaluated metrics, searched documents, generated answers (e.g., results), generated visualizations (e.g., time series visualizations for presentation in a responsive graphical user interface), and / or completed generation.

[0047] A time series is a list of data points ordered in time that can represent changes in the value of data relevant to a particular problem, such as inventory levels, equipment temperature, financial values, or customer transactions, over time. Time series provide historical information that can be analyzed by generative and machine learning algorithms to generate and test predictive models. Example implementations apply cleansing, normalization, aggregation, and combination to time series data to represent the state of a process over time, identifying patterns and correlations that can be used to create and evaluate predictions that can be applied to future behavior.

[0048] Figure 3 A diagram depicts an example network system 300 for generative enterprise search, according to some embodiments. Figure 3 In the example, network system 300 includes enterprise generative artificial intelligence system 302, enterprise systems 304-1 to 304-N (individually enterprise system 304, collectively referred to as enterprise systems 304), external systems 306-1 to 306-N (individually external system 306, collectively referred to as external systems 306), and communication network 308.

[0049] Enterprise generative AI system 302 may include some or all of the functionality described with reference to the enterprise generative AI systems described herein (e.g., enterprise generative AI systems 102 and 400). For example, enterprise generative AI system 400 may utilize generative AI techniques to implement and / or generate intuitive, non-complex interfaces (e.g., Figure 2A 、 Figure 2B 、 Figure 7A and Figure 7B ) to quickly execute complex user requests with improved access, privacy, and security enforcement.

[0050] The enterprise generative AI system 302 can use one or more transformer-based natural language machine learning models to provide real-time and / or near real-time search across multiple different systems. For example, the enterprise generative AI system 302 can use one or more pre-trained autoregressive language models and support distributed transactions. In some embodiments, the enterprise generative AI system provides role-based search access to distributed enterprise datasets. Roles can define access privileges for portions of a distributed dataset, and roles can be set by the enterprise generative AI system 302 based on input provided by one or more users (e.g., administrators and / or users who own or control a portion of the dataset) and / or systems (e.g., machine learning-based systems). For example, a distributed dataset can indicate the level of access and / or usage permissions (e.g., search, access, summary and / or original content, etc.) required for portions of the distributed dataset.

[0051] As discussed elsewhere herein, the enterprise generative artificial intelligence system 302 can provide numerous technical advantages over existing technologies. For example, an input query executed by the system 302 can return results that would not have been identified using traditional search techniques based on the initial user-supplied input query. Additionally, the results themselves can differ from those stored in the distributed dataset (e.g., having a different format, type, or content), and public and private data can be selectively searched without unnecessarily exposing private data.

[0052] As described herein, applications may include, for example, supply chain, CRM, ERP, reliability, defense, sustainability, and / or energy. Artificial intelligence applications are capable of processing data related to real-world systems, devices, and / or scenarios, and analyzing the data to derive one or more insights. Example frameworks for integrating, processing, and abstracting data related to an enterprise logistics optimization development platform may include tools and / or other tools for machine learning, application development and deployment, data visualization, automated control and instruction (such as integration components, data service components, modular service components, and applications that can be located on or behind application modules). The enterprise generative artificial intelligence system 302 can facilitate the design, development, provision, and operation of platforms for industrial-scale applications in various industries (such as the energy industry, health or wearable technology industry, sales and advertising industry, transportation industry, communications industry, scientific and geological research industry, military and defense industry, financial services industry, healthcare industry, manufacturing industry, retail and / or government organizations, etc.). The system can enable the integration and processing of large and highly dynamic data sets from vast networks and large-scale information systems.

[0053] For example, supply chain data may be obtained from a first artificial intelligence application (e.g., an inventory management and optimization or supply chain application), and impact information may be obtained based, at least in part, on the supply chain data and information from another artificial intelligence application (e.g., an artificial intelligence application for monitoring and predicting maintenance needs for a fleet of vehicles). In this example, the supply chain data may indicate a supply issue, the other artificial intelligence application may be used to identify vehicles requiring maintenance, and the impact information may represent insights into how the vehicles requiring maintenance are affected by the supply chain issue (e.g., based on the supply chain data). Example information from different data domains or application objects may include key performance metrics (KPIs) (e.g., from left to right: fleet readiness score, unplanned maintenance avoided over a period of time (hours), number of flights achieved (e.g., due to avoided maintenance), and / or operational risk time), aircraft condition risk score information, component risk score and ranking (e.g., by risk score), information associated with artificial intelligence alerts, flight capability information (e.g., by geographic region), case information, supply chain data, and impact information regarding how aircraft are affected by effects within the supply chain. The ability to obtain insights from one or more artificial intelligence applications and return these insights provides enhanced accessibility functionality. For example, such impact information may not be available solely from a supply chain application or a maintenance application.

[0054] The enterprise system 104 includes enterprise applications (e.g., artificial intelligence applications), enterprise data stores, client systems, and / or other systems of the enterprise information environment. As used herein, an enterprise information environment may include one or more networks (e.g., local, air-gapped, or other) of enterprise systems (e.g., enterprise applications, enterprise data stores) and / or client systems (e.g., computing systems used to access enterprise systems). Enterprise systems include different computing systems, applications, and / or data stores, as well as enterprise-specific requirements and / or features. For example, enterprise systems include access and privacy controls. For example, an organization's private network may include an enterprise information environment that contains various enterprise systems. Enterprise systems include CRM systems, EAM systems, ERP systems, FP&A systems, HRM systems, and / or SCADA systems, among others. Enterprise systems may include or utilize artificial intelligence applications, and artificial intelligence applications may utilize enterprise systems and data. Enterprise systems may include data flows and management of different processes (e.g., of one or more organizations) and may provide access to the enterprise's systems and users while preventing access from other systems and / or users. It will be understood that, in some embodiments, references to an enterprise information environment may also include enterprise systems, and references to enterprise systems may also include an enterprise information environment.

[0055] External systems 306 include applications, data stores, and systems external to the enterprise information environment. In one example, enterprise system 304 may be part of an organization's enterprise information environment that is not accessible to users or systems outside of the enterprise information environment and / or the organization. Thus, example external systems 306 may include Internet-based systems outside of the enterprise information environment, such as news media systems and / or social media systems.

[0056] The communication network 308 may represent one or more computer networks (e.g., LANs, WANs, air-gapped networks, and / or cloud-based networks, etc.) or other transmission media. The communication network 308 may provide communications between the systems, modules, layers, engines, and / or data stores, etc. described herein. In some embodiments, the communication network 308 includes one or more computing devices, routers, cables, buses, and / or other network topologies (e.g., meshes, etc.). In some embodiments, the communication network 308 may be wired and / or wireless. In various embodiments, the communication network 308 may include a local area network (LAN), a wide area network (WAN), the Internet, and / or one or more networks that may be public, private, IP-based, non-IP-based, air-gapped, etc. Although in Figure 1 3. It is not shown, but it will be understood that the communication network 308 or other communication network including some or all of the functionality of the communication network 308 may provide Figure 1 Communication between components.

[0057] Figure 4 A diagram depicts an example of an enterprise generative artificial intelligence system 400 according to some embodiments. Enterprise generative artificial intelligence system 400 can be the same as enterprise generative artificial intelligence system 102 and / or 302. Figure 4 In the example, the enterprise generative artificial intelligence system 400 includes a management module 402, an orchestrator module 404, a retrieval module 406, an agent module 408, a tool module 410, an enterprise understanding module 412, a crawler module 414, an enterprise access control module 416, an artificial intelligence traceability module 418, an extractor module 420, a parallelization module 422, a model generation module 424, a model deployment module 426, a model optimization module 428, a rendering module 430, a communication module 432, a vector data store 440, a registry data store 450 and an enterprise generative artificial intelligence system data store 470.

[0058] Management module 402 can be used to manage (e.g., create, read, update, delete, or otherwise access) data associated with enterprise generative AI system 400. Management module 402 can store or otherwise manage or store data in any of data stores 440-470 and / or one or more other local and / or remote data stores. It will be understood that the data store can be a single data store local to enterprise generative AI system 400 and / or multiple data stores remote from the machine learning-based device. In some embodiments, the data stores described herein include one or more local and / or remote data stores. Management module 402 can operate manually (e.g., by a user interacting with a GUI) and / or automatically (e.g., triggered by one or more of modules 404-432). As with other modules described herein, some or all of the functionality of management orchestrator module 404 can be included in and / or collaborate with one or more other modules, systems, and / or data stores.

[0059] The orchestrator module 404 can be used to generate and / or execute an orchestrator agent (or simply an orchestrator) that can manage, supervise, and / or otherwise orchestrate the components of the enterprise generative artificial intelligence system 400. In some embodiments, the orchestrator module 404 can be used to perform and / or otherwise handle various orchestration (or supervisor) functions. For example, the orchestrator module 404 can enforce conditions (e.g., stopping, resource allocation, and prioritization, etc.). For example, a stopping condition can indicate the maximum number of iterations (or hops) that can be performed before the iterative process terminates. The stopping condition and / or other features managed by the supervisor module can be included in the prompt. The stopping condition ensures that the system will not fall into an endless loop, which allows the system to have the flexibility of having different numbers of iterations for different inputs (e.g., as opposed to having a fixed number of hops). In another example, the supervisor module can perform resource allocation based on computing conditions, such as virtualization or load balancing.

[0060] In an example implementation, orchestrator module 404 creates one or more virtual metadata repositories across data stores, abstracting access to disparate data sources and enabling granular data access control. Orchestrator module 404 can manage a virtual data lake with enterprise catalogs connected to multiple data domains and industry-specific domains. Orchestrator module 404 can create embeddings for multiple data types across multiple industry verticals and knowledge domains, as well as specific enterprise knowledge.

[0061] Embedding objects in the data domain of an enterprise information system enables rapid identification and complex processing using relevance scoring and additional functionality to enforce access, privacy, and security protocols. In some implementations, the orchestrator module 404 can employ various embedding methods and techniques understood by those of ordinary skill in the art. In an example implementation, the orchestrator module can use a model-driven architecture for conceptual representations of enterprise and external data sets and optional data virtualization. For example, the model-driven architecture can be as described in U.S. Patent 10,817,530, serial number 15 / 028,340, issued on October 27, 2020, which claims priority to C3.ai's application entitled "Systems, Methods, and Devices for an Enterprise Internet-of-Things Application Development Platform" dated January 23, 2015. The type system of the model-driven architecture can be used to embed objects in the data domain.

[0062] Type systems provide data accessibility, compatibility, and operability with different systems and data. Specifically, type systems address data operability across various programming languages, inconsistent data structures, and incompatible software application programming interfaces. Type systems provide data abstractions that define extensible type models that enable the dynamic addition of new properties, relations, and functions without expensive development cycles. Type systems can be used as domain-specific languages ​​(DSLs) within a platform that developers, applications, or UIs use to access data. Type systems provide the ability to interact with data to process, predict, or analyze it based on one or more type or function definitions within the type system.

[0063] A type definition can be a canonical type declared in metadata using a syntax similar to that used for types persisted in relational or NoSQL data stores. The canonical model in a type system is an application-agnostic (i.e., application-independent) model, enabling all applications to communicate with each other in a common format. Unlike standard types, a canonical type consists of two parts: a canonical type definition and one or more transformation types. The canonical type defines the interface for integration, and the transformation type is responsible for transforming the canonical type into the corresponding type. Using the transformation type, the integration layer can transform the canonical type into the appropriate type.

[0064] Types can define the metamodel and can be virtual building blocks to create new types, extend existing types, or write business logic on types to dictate how the data in the type will act when called. Logic in the platform can be expressed in JavaScript, which can allow APIs to be used to program against any type in the system. The type system generates composite types that include metadata from multiple layers. Composite types are used to build or generate object instances of specific entities, functions, etc. Composite types can include, for example, entity definitions, application logic functions, and one or more UI view definitions. Composite types can be applied to data stored in one or more databases to create a specific instance of the type that can be used for processing by business logic.

[0065] The orchestrator module 404 can process data received from various data sources in different formats, which can be processed using natural language processing (NLP) (using tokenization, stemming, lemmatization, and / or normalization, etc.) using vectorized data, and can generate pre-trained transformers that are fine-tuned or retrained on specific data tailored to the associated data domain or data application (SaaS application, legacy enterprise application, artificial intelligence application). Further processing can include data modeling feature inspection and / or machine learning model simulation to select one or more appropriate analysis channels. Example data objects can include accounts, products, employees, suppliers, opportunities, contracts, locations, digital portals, geo-located manufacturers, supervisory control and data acquisition (SCADA) information, open manufacturing system (OMS) information, inventory, supply chain, bill of materials, transportation services, maintenance logs, and service logs.

[0066] The orchestrator module 404 can use various components, as needed, to inventory or generate objects (e.g., components, functionality, and / or data) using rich and descriptive metadata to dynamically generate embeddings for developing knowledge across a wide range of data domains (e.g., documents, tabular data, insights derived from artificial intelligence applications, web content, or other data sources). In an example implementation, the orchestrator module 404 can utilize some or all of the components described herein, for example. The orchestrator module 404 can facilitate storage, transformation, and communication to facilitate processing and embedding data. The orchestrator module 404 can create embeddings for multiple data types across multiple industry verticals and knowledge domains, as well as specific enterprise knowledge. Knowledge can be explicitly modeled and / or learned by (one or more) retrieval models. In an example, the orchestrator module generates embeddings that are translated or transformed to be compatible with the knowledge module. The orchestrator module 404 can also be configured to enable different data domains to operate or interface with the components of the enterprise generative artificial intelligence system 400. In one example, orchestrator module 404 can embed objects from a specific data domain and across data domains, applications, data models, analytical byproducts, artificial intelligence predictions, and knowledge repositories to provide robust search functionality without requiring specialized programming for each different data domain or data source. For example, orchestrator module 404 can create multiple embeddings for a single object (e.g., an object can be embedded in a domain-specific or application-specific context). Crawler module 414 (discussed below) in conjunction with orchestrator module 404 can organize the data domain to embed objects from the data domain into an enterprise information system and / or environment.

[0067] The orchestrator module 404 can cause the agent to perform data modeling to transform the original source data format into a target embedding (e.g., object and / or type, etc.). In an example implementation, the orchestrator module 404 and / or the enterprise generative artificial intelligence system 400 typically employs a type system of a model-driven architecture to perform data modeling to transform the original source data format into a target type. The enterprise generative artificial intelligence system 400 and the knowledge base of the generative artificial intelligence model can create the ability to integrate or combine insights from different artificial intelligence applications.

[0068] The retrieval module 406 can be used to receive input, retrieve information from different data sources (e.g., from embedded indexes or vector storage) in different data formats (e.g., data records, documents, database tables, APIs, machine learning-based insights or other application outputs), and output information (e.g., the retrieved information or a subset thereof). In some implementations, the retriever module includes one or more machine learning models, such as deep learning models, neural network models, and transformer-based models. In some implementations, the retriever module can also generate human-readable or machine-readable output.

[0069] Knowledge modeling can be used to train retrieval artificial intelligence models to interact with data sources. Model training can be implemented continuously, asynchronously, with feedback (e.g., re-enforcing learning, etc.). Multiple retrieval models can be trained on a single dataset for different use cases. The retrieval artificial intelligence model is multimodal for different types of data formats (e.g., text, images, and / or videos, etc.), data sources (e.g., databases, repositories, and / or unstructured data, etc.), functions (e.g., enterprise departments and / or technology types, etc.), and access classifications (e.g., licensed, restricted, and / or confidential, etc.). The retrieval artificial intelligence model (or simply the retrieval model) can achieve intelligent identification, access, and understanding for context-specific information management.

[0070] Retrieval models can be trained and reused across environments, and / or retrained for deeper contextual understanding. For example, a retrieval model trained on assessing confidential information can be retrained for organization-specific acronyms, tags, and / or project names to correctly identify and assess access levels. A relevance model with an orchestrator module 404 indexes and crawls data domains to develop embeddings for the retrieval model. The knowledge module creates a knowledge base of inferred ontologies across different data domains. During crawling, the retrieval model collects data and pre-computes paragraph embeddings, which are stored in the knowledge base as a pipeline with traceability to the paragraph location at the data source. One or more retrieval models are used to understand the underlying data, documents, and applications of the enterprise information environment. The underlying data of the enterprise information environment can be embedded by the orchestrator module, for example, using a model-driven architecture for conceptual representation of enterprise and external data sets for data virtualization and access control. Information presented using predictive analytics can include text summaries, ranked results, smart cards, AI-driven chat interfaces, and / or instantly generated content. Predictive analytics can come from one or more artificial intelligence applications, including, for example, supply chain, CRM, ERP, reliability, defense, sustainability, and / or energy.

[0071] During search, the language model is used to understand requests to create one or more queries for the retrieval model to retrieve results. In an example implementation, the retrieval model uses machine learning to return one or more results in the embedding space based on relevance to the query. The enterprise understanding module calls the knowledge base to generate new content based on inferences and insights into relevant data domains. The retrieval model uses the large language model to embed search queries. The Demonstrate-Search-Predict method can be used to allow both the large language model and the retrieval model to understand and generate natural language, allowing them to interact to improve the quality of results.

[0072] Agent module 408 can be used to generate (e.g., spin up) and / or execute agents. In some implementations, agents can be controlled, supervised, and / or orchestrated by orchestrator module 404. In some implementations, agents include reasoning functionality (e.g., via one or more large language models and / or other machine learning models) and control one or more tools (e.g., search tools). In some implementations, agent module 408 performs some or all of the functionality of search module 406.

[0073] Agents can be associated with various tools and domains. This can also allow the enterprise generative artificial intelligence system 400 to more easily and efficiently employ more specific (e.g., domain-specific, industry-specific) models and generate more accurate and reliable results. For example, one agent (or group of agents) can handle requests related to one domain (e.g., aerospace), while another group of agents can handle requests related to a different domain (e.g., defense). This is particularly useful for cross-domain queries (e.g., a single query or set of queries that span multiple domains) and environments that routinely process cross-domain input (e.g., military).

[0074] The tool module 410 can be used to generate and / or execute different types of tools. For example, a tool can include a search tool, a code generation tool, and / or an image generation tool. A tool can include a machine learning model (e.g., a search model) and / or can be controlled by one or more agents. In some implementations, a tool does not include reasoning functionality, but is instead directed by an agent and / or orchestrator.

[0075] The enterprise understanding module 412 can be used to process inputs to determine output results (or "answers"), determine the rationale for the answers, and determine whether the enterprise understanding module 412 requires more information to determine the answers. The enterprise understanding module 412 can output information (e.g., answers or additional queries) in a natural language format. Features of one or more models of the enterprise understanding module 412 define conditions or functions for determining whether more information is needed to satisfy a query, or whether sufficient information exists to satisfy a query and provide an accurate and reliable answer.

[0076] In some implementations, the enterprise understanding module 412 includes one or more large language models. The one or more large language models can be configured to generate and process context and other inputs and outputs described herein. The enterprise understanding module 412 may also include one or more large language models to pre-process queries. For example, the enterprise understanding module 412 can parse complex input into multiple fragments and generate new corresponding queries, which the enterprise understanding module 412 can route to various agents for processing. The enterprise understanding module 412 may also include one or more large language models to process outputs from other models and modules. The enterprise understanding module 412 may also include another large language model for processing results into a format that is more consistent with the answer. The enterprise understanding module 412 may also notify the user and the system if it cannot find an answer (e.g., as opposed to presenting an answer that is likely to be erroneous or biased).

[0077] The enterprise understanding module 412 can generate and store rationales and context. In one example, the rationale is the reasoning that the enterprise understanding module 412 uses to determine an outcome (e.g., a natural language answer, an indication that it needs more information, or an indication that it can satisfy the initial input). In some implementations, the enterprise understanding module 412 can generate context based on the rationale. For example, the context can include a concatenation and / or annotations of one or more data records (or fragments of data records), and / or embeddings associated therewith, and a mapping of the concatenations and / or annotations. For example, the mapping can indicate relationships between different data records (or fragments of data records), and / or weighted or relative values ​​associated with different data records (or fragments), etc.

[0078] In some implementations, the enterprise comprehension module 412 processes inputs to determine one or more results (i.e., outputs, responses, or answers), determines the rationale for the results, and determines whether the enterprise comprehension module 412 requires more information to determine the results. The enterprise comprehension module 412 may output information (e.g., results, new prompts, or additional queries) in a natural language format. In some implementations, features of one or more models of the enterprise comprehension module 412 may define conditions or functions that determine whether more information is needed to satisfy the initial input or whether sufficient information exists to satisfy the initial input. In some implementations, the enterprise comprehension module 412 includes one or more large language models.

[0079] The enterprise understanding module 412 executes queries (e.g., natural language queries) to identify relevant information from the enterprise information environment for presentation to a human-machine interface. In some embodiments, the enterprise understanding module 412 includes a generative artificial intelligence model (e.g., a large language model) that can interact with one or more other models (e.g., a data model, a retrieval model) and a knowledge base to identify key information from relevant data domains and objects in order to respond to requests and generate new insights.

[0080] In some embodiments, enterprise understanding module 412 includes an inference engine that determines the intent of a query and interacts with one or more retrieval models of retrieval module 406 to construct a request to locate and synthesize data. Enterprise understanding module 412 and the retrieval models can perform a series of interactions for complex, multi-level requests to iteratively develop a context-specific construct for the response. For example, in response to a request, enterprise understanding module 412 can infer the category of data required and request a specific retrieval model to retrieve data for the inferred category. Enterprise understanding module 412 can then synthesize the retrieved data and perform further inferences on different categories or data sources, and request a different retrieval model to retrieve data for the further inferred category or data source. The inference, retrieval, and synthesis interaction sequence between the enterprise understanding module and the retrieval AI model can be repeated multiple times (or multiple hops) to develop one or more relevant responses to the request. In some implementations, the orchestrator can control, regulate, or limit the sequence of interactions between the enterprise understanding module and the retrieval models to ensure performance or confidence thresholds. In some implementations, the retrieval models can instruct orchestrator module 404 to perform information virtualization and access control.

[0081] The enterprise understanding module 412 can respond to and / or otherwise process queries (e.g., natural language requests) and provide context-specific information from various enterprise applications. The enterprise understanding module is capable of discerning and deconstructing complex context-specific requests. For example, a natural language request asking "Are my ESG goals on track?" can discern enterprise-specific ESG goals, interpret "on track" as a time-based measure of progress toward the ESG goals, retrieve and process data from enterprise systems including relevant ESG data from artificial intelligence applications, synthesize information that may include images, text, and / or artificial intelligence models, etc. to make one or more determinations, predictions, and generate new content based on the synthesis of the retrieved content to address the request. In some examples, the enterprise generative artificial intelligence system 400 can use external data to supplement or discern and / or synthesize the request.

[0082] The enterprise understanding module 412 can collaborate with the retrieval artificial intelligence model of the retrieval module 406 and learn enterprise-specific context to synthesize information from the knowledge base as well as external data sources. The enterprise understanding module can use a generative large language model to synthesize results from multiple locations. The enterprise understanding module 412 can employ generative artificial intelligence models, large language models, parameter injection, and machine learning techniques. The enterprise understanding module enables response to complex requests (e.g., multi-hop, multi-level, reasoning and / or insights, etc.). The enterprise understanding module can apply time slot technology to deconstruct specific queues or indicators and determine contextual tags to apply specific inferences for synthesis (e.g., enterprise, application, request type, requester type, topic and / or interaction format, etc.).

[0083] The crawling module 414 can be used for intelligent enterprise crawling, and the mapping system can use contextual information (e.g., contextual metadata) and data record embeddings to crawl and index a corpus of data records (e.g., data records of one or more enterprise systems) to provide access control (e.g., role-based access), provide improved data record identification and retrieval, and map relationships between data records. In one example, the contextual information can prevent some users from accessing (e.g., viewing, retrieving) certain data records and improve similarity evaluation used in retrieval operations (e.g., of a generative artificial intelligence process).

[0084] In some implementations, the crawling module 414 can generate embeddings based on data records and / or segments. The embeddings can be generated based on one or more embedding models. In some implementations, the embeddings can be represented by one or more vectors that can be stored in the vector data store 440. The embeddings can be used when retrieving data records and performing similarity or relevance evaluations or other aspects of the retrieval operation.

[0085] Crawling module 414 collaborates with orchestrator module 404 to curate (or "crawl") different data domains to embed objects from the data domains into enterprise generative artificial intelligence system 400 and / or associated enterprise systems and / or data sources, etc. In some embodiments, enterprise understanding module 412 implements and / or utilizes knowledge modeling. In one example, knowledge modeling (or "mapping") understands the enterprise context and understands the semantics and models of enterprise concepts, features, and components. Knowledge modeling can leverage data virtualization capabilities and continuous data processing, as well as artificial intelligence and machine learning techniques. In some embodiments, knowledge is explicitly modeled and / or learned by a retrieval model. Crawling module 414 can couple conceptual domain and industry models (e.g., data models) with the retrieval model of retrieval module 406 to embed and learn task-specific and organization-specific phrases, text, images, abbreviations, acronyms, jargon, etc. Crawling module 414 or its knowledge base can curate or crawl enterprise information systems and develop inferences across different data domains. Knowledge modeling can be performed to organize domain metadata, crawl and index data sources, and employ virtualization to obtain information (eg, data objects). In some implementations, the orchestrator module 404 can be used to embed information and / or data objects of data sources.

[0086] In some embodiments, indexing and embedding operations learn data object semantics to develop enterprise concepts and inferences. Crawl data domains can map databases and structured data to type systems. Knowledge modeling can also track access, security, and privacy characteristics of various information, data objects, and data sources. In some embodiments, crawl module 414 can perform the embedding operations discussed herein.

[0087] Enterprise access control module 416 can be used to provide enterprise access control (e.g., layers and / or protocols) for an enterprise generative artificial intelligence system, associated systems (e.g., enterprise systems), and / or environments (e.g., enterprise information environments). Enterprise access control module 416 can provide functionality for enforcing access control policies regarding the generation of search results and generated search results.

[0088] The enterprise access control module 416 evaluates whether a user is authorized to access all or only a portion of the search results. For example, a user may provide a query associated with a first department or subunit of an organization. Members of that department or subunit may be restricted from accessing certain data, types of data, data models, or other aspects of the data domain in which to search. In the case where the initial search results include data to which the user's access is restricted, the enterprise access control module 416 may determine how to handle such restricted data, such as completely omitting the restricted data, omitting the restricted data but indicating that the search results include data to which the user's access is restricted, or providing information related to all of the initial search results, etc. In an example of completely omitting the restricted data, a final set of search results may be returned for presentation to the user, where the final set of search results does not inform the user that a portion of the initial search results has been omitted. In an example of omitting the restricted data but providing the user with an indication that there is restricted data, the final search results may include only those search results to which the user is authorized to access, but may include information indicating that there were X initial search results but only Y search results were output, where Y < X. In the third example above, all search results, including those to which the user has restricted access, may be output to the user. Additionally or alternatively, the enterprise access control module 416 may communicate with one or more other modules to obtain information that can be used to enforce access permissions / restrictions related to performing the search, rather than for controlling the presentation of results to the user. For example, the enterprise access control module 416 may restrict the data sources to which the search is applied, such as not applying the search to portions of data sources to which the user is denied access and applying the search to portions of data sources to which the user is permitted access, etc. Note that the exemplary techniques described above for enforcing access restrictions related to searching within a search domain have been provided for illustrative purposes and not as a limitation, and it should be understood that a query module operating in accordance with embodiments of the present disclosure may implement other techniques to present search results via an interface based on access restrictions.

[0089] In some embodiments, to facilitate enforcing access restrictions in conjunction with searches performed by the enterprise generative AI system 400, the enterprise access control module 416 may store information associated with the access restrictions or permissions of each user. To retrieve a user's relevant restricted data, the enterprise access control module 416 may receive information identifying the user in conjunction with the input, or when the user logs in to the platform on which the enterprise access control module 416 is executing. The enterprise access control module 416 may use the information identifying the user to retrieve the appropriate restricted data to support the enforcement of access restrictions related to enterprise search. In some embodiments, the enterprise access control module 416 may include credential management functionality for deploying the model-driven architecture of the enterprise generative AI system 400, or may be a remote credential management system communicatively coupled to the enterprise generative AI system 400 via a network.

[0090] The AI ​​traceability module 418 may be used to provide traceability and / or explainability of answers generated by the enterprise generative AI system 400. For example, the AI ​​traceability module 418 may indicate portions of a data record used to generate an answer and its associated data sources.

[0091] The extractor module 420 can be used to process, extract, and / or transform different types of data (e.g., text, database tables, images, videos, and / or code, etc.). For example, the extractor module 420 can take a database table as input and transform it into natural language describing the database table, which can then be provided to the orchestrator module 404, which can then process the transformed input into an "answer" or otherwise satisfy a query. In some implementations, the extractor module 420 includes one or more large language models and one or more other types of machine learning or natural language models. For example, the large language model can be used to process text, while another model can be used to convert (or transform) images, database tables, and / or other non-text input into a text format (e.g., natural language).

[0092] Parallelization module 422 can be used to control the parallelization of the various systems, modules, agents, models, and processes described herein. For example, parallelization module 422 can generate parallel execution of different agents and / or orchestrators. Parallelization module 422 can be controlled by orchestrator module 404.

[0093] The model generation module 424 can be used to generate and / or modify some or all of the different types of models described herein (e.g., machine learning models, large language models, data models). In some implementations, the model generation module 424 can use various machine learning techniques or algorithms to generate the models. As used herein, machine learning can include Bayesian algorithms and / or models, deep learning algorithms and / or models (e.g., artificial neural networks, convolutional neural networks), gap analysis algorithms and / or models, supervised learning techniques and / or models, unsupervised learning algorithms and / or models, random forest algorithms and / or models, similarity learning and / or distance algorithms, and / or generative artificial intelligence algorithms and models, etc.

[0094] The model deployment module 426 can be used to deploy some or all of the different types of models described herein. In some implementations, the model deployment module 426 can deploy models before or after deployment of the enterprise generative artificial intelligence system. For example, the model deployment module 426 can collaborate with the model optimization module 428 to exchange or otherwise change the large language model of the enterprise generative artificial intelligence system. The model optimization module 428 can be used to obtain user feedback (e.g., regarding the final results / answers). For example, the supervisor module can train (or retrain) one or more models of the iterative context-based generative artificial intelligence system based on user feedback.

[0095] In some embodiments, the model optimization module 428 can also change out the models of the iterative context-based generative artificial intelligence system at or during runtime, in addition to before or after runtime. For example, the retriever module and / or the context-based business understanding module can use a particular set of machine learning models for one domain and other models for a different domain. The model exchange module can select and use the appropriate model for a given domain. This can even occur during iterative processing. For example, when the context-based business understanding module generates a new query, the domain can change, which can trigger the model exchange module to select and deploy a different model appropriate for that domain.

[0096] In some embodiments, the model optimization module 428 can train the generative artificial intelligence model to develop different types of responses (e.g., best results, ranked results, smart cards, chatbots, and / or new content generation, etc.).

[0097] The model optimization module 428 can adjust some or all of the models described herein, including the model of the enterprise understanding module (e.g., the large language model). For example, the model optimization module 428 can adjust the generative artificial intelligence model based on tracking user interactions within the system, by capturing explicit feedback (e.g., by training the user interface) and / or implicit feedback, etc. In some example implementations, a reinforcement learning module can optionally be used to accelerate knowledge base guidance. Reinforcement learning can be used to perform explicit guidance of the system using instrumentation such as time spent and results of clicks. Example aspects can include an innovative learning framework that can guide models for different enterprise environments. Example aspects include an innovative learning framework that can guide models for different enterprise environments.

[0098] In some embodiments, the model optimization module 428 can retrain the transformer-based natural language machine learning model periodically, on-demand, and / or in real time. In some example implementations, corresponding candidate transformer-based natural language machine learning models can be trained based on user selections, and the system can replace some or all of the transformer-based natural language machine learning models with one or more candidate transformer-based natural language machine learning models that have been trained on received user selections.

[0099] The presentation module 430 can be used to generate a graphical user interface component (e.g., a server-side graphical user interface component) that can be rendered as a complete graphical user interface on other systems. For example, the presentation module 460 can be used to present an interactive graphical user interface for displaying and receiving information. For example, the presentation module 430 can generate a graphical user interface enterprise search query input and response interface (e.g., as Figure 2A 、 Figure 2B 、 Figure 7A and Figure 7B shown).

[0100] The communication module 432 can be used to send requests, transmit and receive communications, and / or otherwise provide communications with one or more of the systems, modules, engines, layers, devices, data stores, and / or other components described herein. In certain implementations, the communication module 432 can be used to encrypt and decrypt communications. The communication module 432 can be used to send requests to one or more systems and receive data from one or more systems via a network or a portion of a network (e.g., the communication network 308). In certain implementations, the communication module 432 can send requests and receive data over a connection, all or part of which can be a wireless connection. The communication module 432 can request and receive messages and / or other communications from associated systems, modules, and / or layers, etc. The communications can be stored in the enterprise generative artificial intelligence system data store 470.

[0101] Model registry data store 450 can be used to store different models and / or model configurations. For example, the model registry can store different configurations of various large language models (e.g., which can be deployed or exchanged in an enterprise generative artificial intelligence system). Enterprise generative artificial intelligence system 400 includes some or all of the functionality of the enterprise generative artificial intelligence systems described herein (e.g., enterprise generative artificial intelligence systems 102 and / or 302), and / or vice versa.

[0102] Figure 5A flowchart 500 is depicted of an example enterprise generative artificial intelligence method according to some embodiments. In step 502, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 400) generates a set of potential responses to a prompt using one or more data models (e.g., one or more large language models) using data from at least a plurality of data domains of an enterprise information environment including access controls. For example, the set of potential responses includes responses generated based on different data and / or access controls. For example, one of the potential responses may correspond to (e.g., generated for) a supervisor with access to the entire enterprise information environment (e.g., all data domains), and another potential response may correspond to (e.g., generated for) an engineer with limited access to the enterprise information environment (e.g., only one of the data domains). In some embodiments, an enterprise understanding module (e.g., enterprise understanding module 412) generates the potential responses.

[0103] In some embodiments, the one or more data models include multiple models trained for different data domains in the multiple data domains. Each data model can represent corresponding relationships and attributes corresponding to different data domains in the multiple different data domains. The corresponding relationships and attributes can include data types, data formats, and / or industry-specific information. The one or more data models can include one or more large language models. In some embodiments, the selected deterministic response includes predictions, insights, and / or recommendations from one or more artificial intelligence applications.

[0104] In some embodiments, access control enforces restrictions that include at least one of administrative policies, security policies, profile permissions, and organizational controls. Access control can cause different deterministic responses to be selected based on profiles with different access rights. Access control can cause different verification data to be output based on profiles with different access rights. In some embodiments, an access control layer (e.g., enterprise access control layer 115)) and / or an enterprise access module (e.g., enterprise access control module 416)) enforces restrictions, and the enterprise understanding module can select different deterministic responses in view of the restrictions. In another example, the enterprise access control module can select different deterministic responses from potential responses generated by the enterprise understanding module. For example, the enterprise access control module can filter or select deterministic responses based on restrictions.

[0105] In step 504, the enterprise generative artificial intelligence system determines verification data for the set of potential responses. The verification data comes from multiple data domains of the enterprise information environment.

[0106] In step 506, the enterprise generative artificial intelligence system selects a deterministic response from the set of potential responses based on the scoring of the validation data and taking into account the profile information associated with the prompt, as restricted based on access control. In some embodiments, the enterprise comprehension module selects the deterministic response based on instructions from the enterprise access control module. In another example, the enterprise access control module selects the deterministic response from the set of potential responses generated by the enterprise comprehension module.

[0107] In some embodiments, scoring the validation may include determining a plurality of relevance scores associated with at least a portion of each validation data of the potential response set based on one or more data models. A retrieval module (eg, retrieval module 406) may determine the relevance scores.

[0108] In step 508, the enterprise generative artificial intelligence system outputs the selected deterministic response along with the corresponding validation data. In some embodiments, the output includes data visualization, automated controls and instructions, reports, and dynamically configured dashboards. In some embodiments, the enterprise understanding module and / or presentation module (e.g., presentation module 430) outputs the selected deterministic response along with the corresponding validation data.

[0109] Outputs may include predictions, insights, or recommendations from associated AI applications and may take the form of scheduled actions, decision support guidance, inter-platform instruction generation, dynamic dashboards, automatic summary charts of key information, AI-generated summary answers from reference sources, email briefings, alerts, general content generation (e.g., offers), AI-driven chat interfaces, top results lists, etc., from one or more AI applications, open source libraries, document repositories, email systems, etc.

[0110] In step 510, the enterprise generative artificial intelligence system generates a traceability analysis of the validation data, the traceability analysis indicating insights from one or more documents, document fragments, and / or corresponding portions of one or more enterprise datasets. In some embodiments, an artificial intelligence traceability module (e.g., artificial intelligence traceability module 418) generates the traceability analysis.

[0111] In step 512, the enterprise generative artificial intelligence system outputs the traceability analysis. In some embodiments, the artificial intelligence traceability module and / or the presentation module outputs the traceability analysis.

[0112] Figure 6Flowchart 600 depicts an example enterprise generative artificial intelligence method according to some embodiments. In step 602, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 400) receives a query (e.g., from a user or other system). For example, the query can be a natural language query received through a graphical user interface. In some embodiments, an orchestrator module (e.g., orchestrator module 404) receives the query.

[0113] In step 604, the enterprise generative artificial intelligence system identifies one or more enterprise data sets, one or more artificial intelligence applications, and one or more data models from a plurality of different data domains of the enterprise information environment based on the query. In some embodiments, an orchestrator module identifies the enterprise data sets and / or artificial intelligence applications.

[0114] In step 606, the enterprise generative artificial intelligence system determines a plurality of relevance scores associated with at least a portion of the one or more enterprise datasets based on data models from the plurality of different data domains. In some embodiments, a retrieval module (e.g., retrieval module 406) (e.g., using a similarity machine learning model that implements a similarity algorithm) determines the relevance scores.

[0115] In step 608, the enterprise generative artificial intelligence system utilizes one or more generative artificial intelligence models to determine specific information from a plurality of different data domains of the enterprise information environment based on a plurality of relevance scores and one or more enterprise access control protocols. In one example, an enterprise understanding module determines the specific information.

[0116] In step 610, the enterprise generative artificial intelligence system generates natural language output based on specific information from the relevant data domain. In some embodiments, the enterprise understanding module generates the natural language output.

[0117] The enterprise generative artificial intelligence system facilitates presentation of the natural language output at step 612. A presentation module (e.g., presentation module 430) may facilitate presentation (e.g., causing, at least in part, presentation of the natural language output via a graphical user interface of another system).

[0118] In step 614, the enterprise generative artificial intelligence system generates a traceability analysis in natural language output. The traceability analysis can indicate documents, document fragments, and / or insights from corresponding portions of one or more enterprise datasets. In some embodiments, an artificial intelligence traceability module (e.g., artificial intelligence traceability module 418) generates the traceability analysis.

[0119] In step 616, the enterprise generative artificial intelligence system facilitates presentation of the traceability analysis. In some embodiments, a presentation module facilitates presentation of the traceability analysis.

[0120] Figure 7A and Figure 7B A diagram depicts an example enterprise generative artificial intelligence response graphical user interface 700 according to some embodiments. In some embodiments, the enterprise generative artificial intelligence response graphical user interface 700 may be generated, at least in part, by the enterprise generative artificial intelligence system described herein. Figure 7A and Figure 7B In the example of , the enterprise generative artificial intelligence response graphical user interface 700 includes an enterprise search query input portion 702 , a generative enterprise search result portion 705 , and an interactive query portion 708 .

[0121] The enterprise search query input portion 702 presents an enterprise search query 704. In some implementations, the query can be entered through the input portion 702, although it may also have been entered in another interface (e.g., Figure 2A 200 ) and is presented in input portion 702 as part of a generated enterprise search response.

[0122] The generative enterprise search results portion 705 indicates the type of generative artificial intelligence response (e.g., AI summary), planned performance target (e.g., not met), plan (e.g., CO2 reduction plan), performance analysis, and other information. The generative enterprise search results portion 705 includes interactive icons, such as a feedback graphic icon 712. The feedback graphic icon 712 can enable a user to provide feedback (e.g., positive or negative feedback) about the response. The enterprise generative artificial intelligence system can, for example, use the received feedback to improve the enterprise generative artificial intelligence system (e.g., through reinforcement learning). The enterprise search results portion 705 also includes visualizations 714, which can include visualizations of predictive analytics associated with plans and / or planned performance targets. The visualization can be three-dimensional (3D) and include interactive elements associated with deterministic outputs that, for example, enable execution of instructions (e.g., transmission, control system commands, etc.), in-depth traceability, and activation of application features.

[0123] exist Figure 7A and Figure 7B In the example of FIG, the interactive query portion 708 is in a generative artificial intelligence chat interface that provides a summary of the top results using the interactive chat 706. The user can trigger the presentation of the plan details via the interactive graphical icon 716. In some embodiments, the interactive query portion 708 can be triggered by interacting with the portion 710 (e.g., CO 2Reduce Plan icon) to formulate queries to support the top results card. The results section 705 also includes a ranked list 718 of the results, and the user can provide follow-up questions in the interactive input section 722 of the interactive query section 708 and regenerate answers using the regenerate answers interactive graphical icon 720. The interactive query section 708 can enable the user to enter additional related queries (e.g., "follow-up" questions) through the interactive input section 722. Figure 7A and Figure 7B In the example shown, the interactive query portion 708 includes a chat interface, but other interfaces may use a different interactive query portion.

[0124] Figure 8 Diagram 800 depicts an example architecture of an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 102, 302, and / or 400) that implements an iterative enterprise search process, according to some embodiments.

[0125] In particular, enterprise generative AI systems can provide more accurate and less biased results than current generative AI solutions and can effectively leverage information across diverse domains and machine learning-based insights (or simply insights). More specifically, in some embodiments, enterprise generative AI systems can use context-based iteration, which allows models (e.g., machine learning models, large language models) to effectively learn from each other based on the context generated during the iterative process. Thus, in some embodiments, enterprise generative AI systems do not rely solely on general probability distributions and single-shot solutions (e.g., traditional large language model-based systems) and can respond more intelligently to improved validation of inputs than existing generative AI solutions.

[0126] In addition, the enterprise generative artificial intelligence system can also effectively handle complex inputs (e.g., instruction sets, queries, natural language input or other human-readable input, machine-readable input) from users, software (e.g., artificial intelligence applications), and machines. For example, the enterprise generative artificial intelligence system does not rely on a single pass to return a result, but can iteratively communicate between different models and different types of models until an acceptable result (e.g., an "answer") is generated, which is particularly useful for complex inputs. The enterprise generative artificial intelligence system disclosed herein can enhance the quality of the output, improve the likelihood of relevance, reduce the likelihood of presenting erroneous and biased information, and prevent other potential technical problems. For example, supervision functions (e.g., stop conditions) can prevent infinite loops and other technical problems.

[0127] exist Figure 8An example system flow of an enterprise generative AI system that implements an iterative context-based generative AI process is shown in . Figure 8 In the example of FIG, an enterprise generative artificial intelligence system includes one or more retrieval modules 804 and one or more enterprise understanding modules 806. For example, the retrieval module 804 may include one or more machine learning models based on neural network transformers, and the enterprise understanding module 806 may include one or more large language models. These modules and the iterative interaction (e.g., communication) between these modules can allow the enterprise generative artificial intelligence system to achieve the technical features and technical benefits discussed herein.

[0128] exist Figure 8 In the example of FIG, an enterprise generative artificial intelligence system may receive initial input 802 from a user or another system. For example, an orchestrator module 803 may receive input 802. The enterprise generative artificial intelligence system may provide this input to a retrieval module 804, which may then obtain and "retrieve" information from various enterprise data sources (e.g., data stores, databases, and / or artificial intelligence applications, etc.). The enterprise generative artificial intelligence system may use this information to generate an initial prompt for an enterprise understanding module 806. The enterprise understanding module 806 may process the initial prompt and determine, based on the initial input, whether it has sufficient information to meet the criteria (e.g., answer the question). If it has sufficient information to meet the initial input, the enterprise understanding module may then provide the result to a recipient, such as the user or system that provided the initial input. However, if the enterprise understanding module 806 determines, based on the initial input, that it does not have sufficient information to meet the criteria, it may further synthesize information through an iterative process that provides the core benefits of the system.

[0129] There are many reasons why the enterprise comprehension module 806 may need additional information. For example, traditional systems use a single pass that only solves a portion of a complex input. Enterprise generative artificial intelligence systems address this issue by triggering subsequent iterations to solve for the remaining portions of the complex input and including context to further refine the pass.

[0130] More specifically, if the enterprise understanding module 806 determines that it needs additional information to satisfy the initial input, it can generate context-specific data (or simply "context") that will inform future iterations of the process and help the system satisfy the initial input more efficiently and accurately. The context is based on the rationale used by the enterprise understanding module 806 when it is processing the query (or other input). For example, the enterprise understanding module 806 can receive a snippet retrieved by the retrieval module 804. For example, the snippet can be a paragraph (one or more) of a data record, and the snippet can be associated with an embedding from the embedding data store 808, which facilitates processing by the enterprise understanding module 806. The query and rationale generator 812 of the enterprise understanding module 806 can process this information and generate a rationale for why it produced the result it produced. This rationale can be stored by the enterprise generative artificial intelligence system in the historical rationale data store 810 and provide a basis for subsequent iterations of the context.

[0131] Subsequent iterations may include the enterprise understanding module 806 generating a new query, request, or other output, which is then passed back to the retrieval module 804. The retrieval module 804 may process the new query and retrieve additional information. The system then generates a new prompt based on the additional information and context. The enterprise understanding module 806 may process the new prompt and again determine whether it requires additional information. If it requires additional information (e.g., as shown in step 807), the enterprise generative artificial intelligence system may repeat (e.g., iterate) the process until the enterprise understanding module 806 can satisfy the criteria based on the initial input (e.g., the query), at which point the enterprise understanding module 806 may generate (step 813) an output result 814 (e.g., "answer" or "I don't know"). For example, if no relevant paragraphs and / or not enough relevant paragraphs are generated (e.g., by applying rules), then generating the answer "I don't know" may prevent hallucinations and increase performance related to "I don't know" questions while saving calls to the model (e.g., a large language model).

[0132] In some embodiments, whether sufficient information exists and / or is associated with sufficient information can be determined based on the number of paragraphs that were retrieved but from which relevant information was not extracted (e.g., using business comprehension module 806 and / or extractor module 420). For example, a threshold number or percentage of retrieved paragraphs from which relevant information was extracted (e.g., a particular number or percentage of retrieved paragraphs) may need to be met for business comprehension module 806 to determine that they have sufficient information to answer the query. In another example, a threshold number or percentage of retrieved paragraphs from which relevant information was not extracted (e.g., 4 paragraphs or 80% of retrieved paragraphs) may cause business comprehension module 806 to determine that they do not have sufficient information to answer the query.

[0133] The enterprise generative AI system may also implement supervisory functions, such as stopping conditions that prevent the system from hallucinating or otherwise providing incorrect answers. Stopping conditions may also prevent the system from executing infinite iteration loops. In one example, the enterprise generative AI system may limit the number of iterations that can be performed before the enterprise comprehension module 806 provides an output result or indicates that no output result can be found. Users may also provide feedback 816 that is stored in the feedback data store 818. In some embodiments, the enterprise generative AI system may use feedback to improve the accuracy and / or reliability of the system.

[0134] Figure 9 Flowchart 900 depicts an example enterprise generative artificial intelligence method according to some embodiments. In step 902, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 400) processes a query input. Processing the query input may include examining data modeling features and / or simulating a machine learning model to select one or more analysis channels. In some embodiments, an orchestrator module (e.g., orchestrator module 404) processes the query input.

[0135] In step 904, the enterprise generative artificial intelligence system identifies one or more artificial intelligence applications and a plurality of associated data models from a plurality of data domains based on the query input. In some embodiments, the orchestrator module identifies the artificial intelligence applications and data models.

[0136] In step 906, the enterprise generative artificial intelligence system analyzes the query input based on data models from multiple data domains. In some embodiments, an orchestrator module analyzes the query input.

[0137] In step 908, the enterprise generative artificial intelligence system determines a relevance score based on analysis of the query input by the machine learning model. In some embodiments, a retrieval module (e.g., retrieval module 406)) determines the relevance score.

[0138] In step 910, the enterprise generative AI system generates one or more query sets based on the relevance scores for execution on at least one of the one or more AI applications. The query sets can be configured to address a range of different AI applications and datasets. The one or more query sets configured for the range of different AI applications and datasets are based on a type system to define types for use by the different AI applications. In some embodiments, a retriever module generates the one or more query sets.

[0139] In step 912, the enterprise generative artificial intelligence system composes a response output based on the results of executing the one or more generated query sets. The response output may include a report or a dynamically configured dashboard based on any of the predictions, insights, and / or recommendations from the associated artificial intelligence applications. The response output may include a report or a dynamically configured dashboard based on the predictions, insights, and / or recommendations from the associated artificial intelligence applications. The results of the one or more generated query sets (e.g., the execution of these query sets) may involve multiple different artificial intelligence applications, and the results may include newly generated insights from multiple different artificial intelligence applications. In some embodiments, an enterprise understanding module (e.g., enterprise understanding module 412) composes the response.

[0140] Figure 10 Flowchart 1000 depicts an example enterprise generative artificial intelligence method according to some embodiments. In step 1002, an enterprise generative artificial intelligence system (e.g., enterprise generative artificial intelligence system 400) receives an enterprise search query. The enterprise search query can be natural language input. In some embodiments, an orchestrator module (e.g., orchestrator module 404) receives the query.

[0141] In step 1004, the enterprise generative artificial intelligence system retrieves a plurality of data records associated with at least a portion of enterprise data of the enterprise information environment based on the enterprise search query and one or more retriever models. In some embodiments, a retrieval module (e.g., retrieval module 406) retrieves the data records.

[0142] In step 1006, the enterprise generative artificial intelligence system determines corresponding relevance scores for the retrieved data records using one or more retriever models. In some embodiments, the retrieval model determines the relevance scores.

[0143] In step 1008, the enterprise generative artificial intelligence system selects at least one of the retrieved data records based on the corresponding relevance score. In one example, the retrieval model selects the retrieved document. In another example, the enterprise understanding module (e.g., enterprise understanding module 412) selects the document.

[0144] In step 1010, the enterprise generative artificial intelligence system selects at least a portion of at least one of the retrieved data records based on one or more enterprise access control protocols. A retriever module, an enterprise understanding module, and / or an enterprise access control module (e.g., enterprise access control module 416) selects the portion of the retrieved data record.

[0145] In step 1012, the enterprise generative artificial intelligence system determines enterprise access-controlled natural language output based on the at least one selected portion using one or more large language models. In some embodiments, the enterprise understanding module determines the enterprise access-controlled natural language output.

[0146] In step 1014, the enterprise generative artificial intelligence system facilitates presentation of enterprise-access-controlled natural language output. In some embodiments, a presentation module (e.g., presentation module 430) facilitates presentation of enterprise-access-controlled natural language output.

[0147] Figure 11 Schematic diagram 1100 depicts an example of a computing device 1102. Any of the systems, modules, layers, engines, data stores, and / or networks described herein may include one or more instances of computing device 1102. In some embodiments, the functionality of computing device 1102 is modified to perform some or all of the functionality described herein. Computing device 1102 includes a processor 1104, memory 1106, storage 1108, input device 1110, communication network interface 1112, and output device 1114 communicatively coupled to a communication channel 1116. Processor 1104 is configured to execute executable instructions (e.g., a program). In some embodiments, processor 1104 includes circuitry or any processor capable of processing executable instructions.

[0148] Memory 1106 stores data. Some examples of memory 1106 include storage devices such as RAM, ROM, RAM cache, virtual memory, etc. In various embodiments, working data is stored in memory 1106. Data in memory 1106 can be cleared or eventually transferred to storage 1108.

[0149] Storage 1108 includes any storage configured to retrieve and store data. Some examples of storage 1108 include a flash drive, a hard drive, an optical drive, cloud storage, and / or magnetic tape. Each of memory system 1106 and storage 1108 includes a computer-readable medium that stores instructions or programs executable by processor 1104.

[0150] Input device 1110 is any device for inputting data (e.g., a mouse and keyboard). Output device 1114 outputs data (e.g., a speaker or display). It will be understood that storage 1108, input device 1110, and output device 1114 may be optional. For example, a router / switch may include processor 1104 and memory 1106 as well as devices for receiving and outputting data (e.g., communication network interface 1112 and / or output device 1114).

[0151] The communication network interface 1112 can be coupled to a network (e.g., network 308) via a link 1118. The communication network interface 1112 can support communication via an Ethernet connection, a serial connection, a parallel connection, and / or an ATA connection. The communication network interface 1112 can also support wireless communications (e.g., 802.11a / b / g / n, WiMax, LTE, Wi-Fi). It will be apparent that the communication network interface 1112 can support many wired and wireless standards.

[0152] It will be understood that the hardware elements of the computing device 1102 are not limited to Figure 11 . The computing device 1102 may include more or fewer hardware, software, and / or firmware components than those depicted (e.g., drivers, operating systems, touch screens, and / or biometric analyzers, etc.). Furthermore, hardware elements may share functionality and still be within the various embodiments described herein. In one example, encoding and / or decoding may be performed by the processor 1104 and / or a coprocessor located on a GPU (e.g., NVidia).

[0153] Example types of computing devices and / or processing devices include one or more microprocessors, microcontrollers, reduced instruction set computers (RISCs), complex instruction set computers (CISCs), graphics processing units (GPUs), data processing units (DPUs), virtual processing units, associative processing units (APUs), tensor processing units (TPUs), vision processing units (VPUs), neuromorphic chips, AI chips, quantum processing units (QPUs), brain wafer-scale engines (WSEs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or discrete circuit systems.

[0154] It will be understood that a “module,” “layer,” “engine,” “system,” “data store,” and / or “database” may include software, hardware, firmware, and / or circuitry. In one example, one or more software programs comprising instructions executable by a processor may perform one or more of the functions of the modules, layers, engines, data stores, databases, or systems described herein. In another example, the circuitry may perform the same or similar functions. Alternative embodiments may include more, fewer, or functionally equivalent modules, layers, systems, data stores, or databases and still be within the scope of the present embodiments. For example, the functionality of the various systems, modules, layers, engines, data stores, and / or databases may be combined or divided differently. The data store or database may include cloud storage. It will be further understood that the term “or,” as used herein, may be interpreted in an inclusive or exclusive sense. In addition, multiple instances may be provided for a resource, operation, or structure described herein as a single instance.

[0155] The data stores described herein may be of any suitable structure (e.g., active databases, relational databases, self-referential databases, tables, matrices, arrays, flat files, document-oriented storage systems, and / or non-relational No-SQL systems, etc.), and may be cloud-based or otherwise. The claimed solutions, rooted in computer technology, overcome problems arising specifically in the field of computer technology.

[0156] Large language models are trained using large amounts of data that constitute the model's inherited knowledge. Large language models typically utilize this inherited knowledge in vector storage to generate word strings that conform to lexical and grammatical rules, but lack accuracy or validity. Hallucinations in artificial intelligence refer to AI-generated responses that contain non-responsive, false, or misleading information. AI hallucinations are associated with irrational responses or beliefs.

[0157] The enterprise AI search exposes the LLM to a corpus of data to generate responses that are rooted in the documents or data. The responses generated by the LLM can be reviewed against the corpus of documents or data to identify true references that validate the responses as hallucination-resistant. In some embodiments, the generative artificial intelligence model is separated from the enterprise data of the enterprise information environment, and the separation includes the generative artificial intelligence model not being trained on the enterprise data of the enterprise information environment. In some embodiments, the natural language output includes a deterministic response that is at least partially caused by the separation of the generative artificial intelligence model from the enterprise data of the enterprise information environment. In some embodiments, the separation of the generative artificial intelligence model from the enterprise data of the enterprise information environment reduces hallucinations and information leakage of the generative artificial intelligence model relative to other generative artificial intelligence models that have been trained on other enterprise data of other enterprise information environments.

[0158] In the flowcharts and / or sequence diagrams, the flowcharts illustrate a sequence of steps by way of example. It should be understood that some or all of the steps may be repeated, reorganized to be performed in parallel, and / or reordered, where applicable. In addition, for the sake of clarity, some steps that could have been included may have been removed to avoid providing too much information, and some steps that were included may have been removed but may be included for the sake of clarity.

[0159] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to generate a set of potential responses to a prompt using one or more data models using data from at least a plurality of data domains of an enterprise information environment including access controls. Validation data is determined for the set of potential responses, where the validation data is from the plurality of data domains of the enterprise information environment. A deterministic response is selected from the set of potential responses based on a score for the validation data and restrictions based on access controls taking into account profile information associated with the prompt. The selected deterministic response is output along with the corresponding validation data.

[0160] In some embodiments, the system includes one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to perform the functions described herein. The system, method, and non-transitory computer-readable medium are also configured to generate a traceability analysis of the verification data, the traceability analysis indicating any of documents, document fragments, and insights for corresponding portions of one or more enterprise data sets. The system, method, and non-transitory computer-readable medium are also configured to determine a plurality of relevance scores associated with at least a portion of each verification data of a potential response set based on one or more data models. The one or more data models include a plurality of models trained for different data domains in a plurality of data domains, each data model representing corresponding relationships and attributes of corresponding different data domains in the plurality of different data domains, and the corresponding relationships and attributes include any of data types, data formats, and industry-specific information.

[0161] In some embodiments, the data model is a large language model. In some embodiments, access control enforces restrictions that include at least one of management policies, security policies, profile permissions, and organizational controls. In some embodiments, access control enables selection of different deterministic responses based on profiles with different access rights. In some embodiments, access control enables output of different verification data based on profiles with different access rights. In some embodiments, the selected deterministic response includes at least one of a prediction, an insight, and a recommendation from an artificial intelligence application. In some embodiments, the output includes at least one of: data visualization, automated controls and instructions, reports, and dynamically configured dashboards.

[0162] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to receive a query. Based on the query, one or more enterprise datasets, one or more artificial intelligence applications, and one or more data models from a plurality of different data domains in an enterprise information environment are identified. Based on the data models from the plurality of different data domains, a plurality of relevance scores associated with at least a portion of the one or more enterprise datasets are determined. Specific information from the plurality of different data domains in the enterprise information environment is determined by one or more generative artificial intelligence models based on the plurality of relevance scores and one or more enterprise access control protocols. Natural language output is generated based on the specific information from the relevant data domains, and presentation of the natural language output is facilitated.

[0163] In some embodiments, the system includes one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to perform the functions described herein. In some embodiments, the query includes a natural language query received through a graphical user interface. In some embodiments, the one or more enterprise datasets include any of documents, document fragments, and insights generated by one or more artificial intelligence applications. In some embodiments, each relevance score is associated with a corresponding portion of the one or more enterprise datasets, and wherein each relevance score is determined relative to other corresponding portions of the one or more enterprise datasets. In some embodiments, the systems, methods, and non-transitory computer-readable media are further configured to perform a traceability analysis that generates a natural language output, the traceability analysis indicating any of documents, document fragments, and insights of the corresponding portions of the one or more enterprise datasets.

[0164] In some embodiments, each data model in the plurality of data models corresponds to a different data domain in the plurality of different data domains. In some embodiments, each data model represents corresponding relationships and attributes corresponding to the different data domain in the plurality of different data domains. In some embodiments, the corresponding relationships and attributes include any of data types, data formats, and industry-specific information. In some embodiments, the natural language output includes a summary of at least one of the corresponding portions of the one or more enterprise datasets associated with the relevance score.

[0165] In some embodiments, the systems, methods, and non-transitory computer-readable media are further configured to embed the corresponding objects in multiple different data domains of the enterprise information environment. The corresponding objects can enable one or more enterprise access control protocols. In some embodiments, the enterprise access control protocol includes an enterprise access control protocol based on user roles. In some embodiments, the enterprise access control protocol enables a first user having a first user role to be presented with different natural language output relative to a second user having a second user role. In some embodiments, the enterprise access control protocol enables and / or prevents the presentation of at least a portion of the natural language output. In some embodiments, the enterprise access control protocol prevents at least a portion of specific information from the relevant data domain from being used to generate the natural language output; and, prior to identification, prevents access to any of a specific enterprise data set, a specific artificial intelligence application, a specific data model, and a specific data domain of the enterprise information environment.

[0166] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to process a query input. Based on the query input, one or more artificial intelligence applications and a plurality of associated data models from a plurality of data domains are identified. The query input is analyzed based on the data models from the plurality of data domains. A relevance score is determined by a machine learning model based on the analysis of the query input. Based on the relevance score, one or more query sets are generated for execution on at least one of the one or more artificial intelligence applications. A response output is constructed based on results of executing the one or more generated query sets.

[0167] In some embodiments, the system includes one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to perform the functions described herein. In some embodiments, processing the query input includes any of data modeling feature inspection and machine learning model simulation to select one or more analysis channels. In some embodiments, the response output includes any of predictions, insights, and recommendations from the associated artificial intelligence application. In some embodiments, the response output includes any of a report or a dynamically configured dashboard based on any of the predictions, insights, or recommendations from the associated artificial intelligence application. The one or more query sets for running on the associated artificial intelligence application are configured to address a range of different artificial intelligence applications and data sets. In some embodiments, the one or more query sets configured for a range of different artificial intelligence applications and data sets define types for use by the different artificial intelligence applications based on a type system.

[0168] In some embodiments, the results of one or more generated query sets relate to multiple different artificial intelligence applications, and wherein the results may include newly generated insights from the multiple different artificial intelligence applications. In some embodiments, processing the query input includes natural language processing with vectorized data, and a pre-trained transformer fine-tuned or re-trained on specific data tailored to the query type of the associated artificial intelligence application may be generated.

[0169] Various embodiments of the present disclosure include systems, methods, and non-transitory computer-readable media configured to receive an enterprise search query, wherein the enterprise search query comprises natural language input. Based on the enterprise search query and one or more retriever models, a plurality of data records associated with at least a portion of enterprise data of an enterprise information environment are retrieved. For each of the retrieved data records, a corresponding relevance score is determined using the one or more retriever models. At least one of the retrieved data records is selected based on the corresponding relevance score. At least a portion of at least one of the retrieved data records is selected based on one or more enterprise access control protocols.

[0170] The systems, methods, modules, layers, engines, data stores, and / or databases described herein may be, at least in part, processor-implemented, wherein one or more specific processors are examples of hardware. For example, at least a portion of the operations of the methods may be performed by one or more processors or processor-implemented modules. In addition, the one or more processors may also operate to support the performance of related operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least a portion of the operations may be performed by a computer group (as an example of a machine including a processor), wherein the operations may be accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application program interfaces (APIs)).

[0171] According to an example disclosed herein, a computer-implemented method is provided, comprising: generating a set of potential responses to a prompt using one or more data models using data from at least multiple data domains of an enterprise information environment including access controls; determining verification data for the set of potential responses, wherein the verification data comes from multiple data domains of the enterprise information environment; selecting a deterministic response from the set of potential responses based on a score of the verification data and based on restrictions of access controls associated with the prompt; and outputting the selected deterministic response together with the corresponding verification data.

[0172] The validation data may include or otherwise indicate data from a plurality of data domains upon which the set of potential responses is based. For example, the validation data may include or otherwise indicate any of documents, document fragments, and insights for a corresponding portion of the data domain upon which the potential responses are based. For example, for each potential response, the validation data for the potential response may include or otherwise indicate data from a plurality of data domains upon which the potential response is based (e.g., any of documents, document fragments, and insights for a corresponding portion of one or more enterprise datasets).

[0173] Determining validation data and outputting the validation data along with the definitive response provides traceability and transparency regarding the data used to provide the response to the prompt, thereby improving the integrity of the response. For example, the validation data allows the response to be checked against the information source on which it is based. The validation data also allows transparency regarding the operation of one or more data models and provides insight into the internal functionality of the data models.

[0174] Access controls may include controls that take into account profile information associated with the prompt. Access controls may include controls associated with credentials associated with input of the prompt (e.g., user credentials and / or user profile). For example, the prompt may be input at a computing device by a user logging in or otherwise accessing a system or device using credentials, such as user credentials (e.g., login credentials). The credentials used when inputting the prompt may have access controls associated with them. Access controls may include permissions or permissions associated with the credentials and may be used to restrict access to data from multiple data domains. For example, access controls may be used to determine data from multiple data domains that are authorized to access and / or data from multiple data domains that are not authorized to access. Selecting a deterministic response from the set of potential responses based on the scoring of the validation data and restrictions based on the access controls may include not selecting a deterministic response from the set of potential responses that includes information that is restricted based on the access controls. For example, one or more potential responses in the set of potential responses may be based on data from multiple data domains, and the access controls indicate that the data should not be included in the response to the prompt (e.g., it may include information that is restricted for the user inputting the prompt). Access to such potential responses may be limited and may not be offered as the selected deterministic response.

[0175] Selecting a deterministic response from the set of potential responses based on the score of the validation data and the restrictions based on the access control may include selecting a deterministic response from the set of potential responses that includes information that is not restricted based on the access control. For example, one or more potential responses in the set of potential responses may be based on data from multiple data domains, and the access control indicates that the data may be included in the response to the prompt (e.g., it does not include information that is restricted to the user entering the prompt).

[0176] Validation data and access controls can be used together to select a deterministic response from a set of potential responses. For example, validation data can be used to identify data (e.g., any of documents, document fragments, and insights from corresponding portions of one or more enterprise datasets) from multiple data domains on which potential responses and access controls can be used to determine whether access to potential responses based on the data should be permitted or restricted.

[0177] Thus, validation data and access controls can be used to provide a synergistic technical effect of improving the security and integrity of the enterprise information environment. This is different from other model-based (e.g., generative AI-based) response systems that may be subject to information leakage when information and data are ingested into the model and / or during the training of the model without or with limited traceability or security controls.

[0178] Deterministic responses can include responses that are reproduced multiple times in response to the same prompt and access controls. This differs from other model-based (e.g., generative AI-based) response systems that can generate different responses for different instances of the same prompt being presented and subject to the same access controls. Deterministic responses improve the integrity, reliability, and traceability of responses provided in response to prompts.

[0179] Outputting the selected deterministic response along with the corresponding verification data may include displaying a representation of the selected deterministic response and the corresponding verification data on an electronic display. Outputting the selected deterministic response along with the corresponding verification data may include transmitting the selected deterministic response along with the corresponding verification data to a computing device for display at the computing device.

[0180] The selected deterministic response and the corresponding verification data outputted depend on the technical functionality of the one or more data models. The one or more data models may have been trained using a machine learning algorithm. Thus, the operation of the one or more data models in generating a set of potential responses may be based on parameters that the one or more data models have learned through training (rather than parameters set by a human programmer). The one or more data models may be implemented in dedicated hardware. Additionally or alternatively, the one or more data models may include simulating the one or more data models in software.

[0181] One or more data models may have been configured through training based on training data. The training data may be different from data from at least a plurality of data domains in the enterprise information environment. In other words, one or more data models may not have been trained on data stored in the enterprise information environment. This can prevent information about data stored in a data domain in the enterprise information environment from being leaked into the output of one or more data models, thereby improving the security and integrity of the data domain.

[0182] The method may also include generating a traceability analysis of the verification data, the traceability analysis indicating any of documents, document fragments, and insights from corresponding portions of the one or more enterprise datasets. The method may also include outputting the generated traceability analysis. For example, the generated traceability analysis may be transmitted to a computing device for display at the computing device. Scoring the verification data may include determining a plurality of relevance scores associated with at least a portion of each verification data for the set of potential responses based on the one or more data models.

[0183] The one or more data models may include multiple models trained for different ones of the multiple data domains, wherein each data model represents corresponding relationships and attributes of a corresponding different one of the multiple different data domains, and wherein the corresponding relationships and attributes include any of data types, data formats, and industry-specific information.

[0184] The data model may be a large language model. Access control may enforce restrictions comprising at least one of management policy, security policy, profile permissions, and organizational controls. Access control may cause different deterministic responses to be selected based on profiles with different access permissions. Access control may cause different verification data to be output based on profiles with different access permissions. The selected deterministic response may include at least one of a prediction, insight, and recommendation from the artificial intelligence application. The output may include at least one of: data visualization, automated controls and instructions, reports, and dynamically configured dashboards.

[0185] According to an example disclosed herein, a computer-implemented method is provided, comprising: receiving a query; identifying, based on the query, one or more enterprise data sets, one or more artificial intelligence applications, and one or more data models from multiple different data domains of an enterprise information environment; determining, based on the data models from the multiple different data domains, multiple relevance scores associated with at least a portion of the one or more enterprise data sets; determining, using one or more generative artificial intelligence models, specific information from multiple different data domains of the enterprise information environment based on the multiple relevance scores and one or more enterprise access control protocols; generating natural language output based on the specific information from the relevant data domains; and facilitating presentation of the natural language output.

[0186] Facilitating presentation of the natural language output may include generating a representation of the natural language output for display on an electronic display. Facilitating presentation of the natural language output may include displaying the representation of the natural language output on an electronic display. Facilitating presentation of the natural language output may include transmitting the natural language output to a computing device for display at the computing device.

[0187] The one or more enterprise access control protocols may include a control protocol associated with credentials (e.g., user credentials and / or user profile) associated with input of a query. For example, a query may be input at a computing device by a user who logs in or otherwise accesses a system or device using credentials, such as user credentials (e.g., login credentials). The credentials used when inputting the query may have access controls associated with them. The access controls may include permissions or permissions associated with the credentials and may be used to restrict access to data from multiple data domains. For example, the access controls may be used to determine data from multiple data domains that is authorized to be accessed and / or data from multiple data domains that is not authorized to be accessed.

[0188] Determining specific information from the plurality of data domains may include determining information that is restricted based on access controls. For example, access controls may indicate that some information from the plurality of data domains should not be included in a response to a query (e.g., it may include information that is restricted to the user who entered the query). Access to such information may be restricted and may not be determined to be part of the specific information.

[0189] Determining specific information from the plurality of data domains may include determining information that includes information that is not restricted based on an access control. For example, the access control may indicate that some information from the plurality of data domains may be included in the response to the query (e.g., it may not include information that is restricted to the user entering the query).

[0190] Access controls can be used to provide a technical effect of improving the security and integrity of the enterprise information environment. This is different from other model-based (e.g., generative AI-based) response systems that may be subject to information leakage when information and data are ingested into the model and / or during the training of the model with no or limited traceability or security controls.

[0191] The natural language output based on specific information depends on the technical functionality of one or more data models and / or one or more generative artificial intelligence models. One or more data models and / or one or more generative artificial intelligence models may have been trained using a machine learning algorithm. Therefore, the operation of one or more data models and / or one or more generative artificial intelligence models in determining specific information and / or natural language output can be based on parameters that one or more data models and / or one or more generative artificial intelligence models have learned through training (rather than parameters that have been set by human programmers). One or more data models and / or one or more generative artificial intelligence models can be implemented in dedicated hardware. Additionally or alternatively, one or more data models and / or one or more generative artificial intelligence models may include simulating one or more data models and / or one or more generative artificial intelligence models in software.

[0192] One or more data models and / or one or more generative AI models may have been configured through training based on training data. The training data may be different from the data from the one or more enterprise datasets. That is, the one or more data models and / or one or more generative AI models may not have been trained on the data stored in the one or more enterprise datasets. This can prevent information leakage from the output of the one or more data models to the data stored in the one or more enterprise datasets, thereby improving the security and integrity of the enterprise datasets.

[0193] The query may include a natural language query received through a graphical user interface. The one or more enterprise datasets may include any of documents, document fragments, and insights generated by one or more artificial intelligence applications. Relevance scores may be associated with respective portions of the one or more enterprise datasets, and each relevance score may be determined relative to other respective portions of the one or more enterprise datasets.

[0194] Note that the above functionality has been provided by way of non-limiting examples, and other techniques may be used to generate queries and commands. For example, in additional or alternative implementations, a multimodal or generative pre-trained transformer (which is an autoregressive language model that uses deep learning to produce human-like text) may be used to generate queries based on the search input (i.e., without using a seed library). For illustration, assume the search input is "What is the riskiest motor actuator at Tinker?" As in the above techniques, the search input may be subjected to entity extraction and mapping, which will result in an entity-matched search input "What is the riskiest WUC 11AAF at location AFB0123?" The input is subjected to embedding and vectorization, where a large language model is used for query generation, and the entity-matched search input may be provided to a generative multimodal or large language model algorithm to generate a query. In such an implementation, contextual information, such as a schema defining metadata for table titles, field descriptions, and join keys, may be provided to the generative multimodal algorithm, which may be used to retrieve search results. For example, a schema may be used to transform the entity-matched search input into a query (e.g., an SQL query).

[0195] The system can leverage features of model-driven architectures that use rich and descriptive metadata to represent system objects (e.g., components, functionality, data, etc.) to dynamically generate queries for searching across a wide range of data domains (e.g., documents, tabular data, insights derived from AI applications, web content, or other data sources). Furthermore, the query module of an embodiment may be particularly well-suited for searches involving type systems whose objects are exposed through metadata. Thus, the query module disclosed herein represents an improvement in the field of query modules and provides a mechanism for rapidly deploying search functionality across various data domains relative to existing query modules, which are typically limited in their searchable data domains (e.g., a web query module is limited to web content, a file system query module is limited to searches of a file system, etc.).

[0196] The method may also include determining validation data for the specific information from a plurality of different data domains. The validation data may include or otherwise indicate data from one or more enterprise datasets upon which the determined specific information is based. For example, the validation data may include or otherwise indicate any of documents, document fragments, and insights from corresponding portions of the one or more enterprise datasets upon which the potential response is based.

[0197] Determination of validation data can provide traceability and transparency about the data about which natural language output based on specific information is provided, thereby improving the integrity of the output. For example, validation data can allow the output to be checked against the information source on which it is based. Validation data can also allow transparency about the operation of one or more data models and / or one or more generative artificial intelligence models, and can allow insight into the internal functioning of the model.

[0198] The method may also include outputting the determined verification data. Outputting the determined verification data may include displaying a representation of the verification data on an electronic display and / or transmitting the verification data to a computing device for display at the computing device. The method may also include generating a traceability analysis in natural language output, the traceability analysis indicating any of documents, document fragments, and insights for corresponding portions of one or more enterprise datasets. The traceability analysis may be based on the determined verification data. The method may also include outputting the generated traceability analysis. For example, the generated traceability analysis may be transmitted to a computing device for display at the computing device.

[0199] Each data model in the plurality of data models may correspond to a different data domain in a plurality of different data domains. Each data model may represent corresponding relationships and attributes corresponding to the different data domains in the plurality of different data domains. The corresponding relationships and attributes may include any of data types, data formats, and industry-specific information. The natural language output may include a summary of at least one of the corresponding portions of the one or more enterprise datasets associated with a relevance score. The method may also include embedding corresponding objects in the plurality of different data domains of the enterprise information environment; and wherein the corresponding objects enable one or more enterprise access control protocols.

[0200] The enterprise access control protocol may include a user role-based enterprise access control protocol. The enterprise access control protocol may cause a first user having a first user role to be presented with different natural language output relative to a second user having a second user role. The enterprise access control protocol may cause any of the following actions: preventing at least a portion of the natural language output from being presented; preventing at least a portion of specific information from a related data domain from being used to generate the natural language output; and blocking access to any of a specific enterprise data set, a specific artificial intelligence application, a specific data model, and a specific data domain of the enterprise information environment prior to identification.

[0201] The generative artificial intelligence model can be separated from the enterprise data of the enterprise information environment, and the separation can include the generative artificial intelligence model not being trained on the enterprise data of the enterprise information environment. The generative artificial intelligence model can be trained on domain data associated with a different domain, where the domain data does not include the enterprise data of the enterprise information environment.

[0202] The natural language output may include a deterministic response caused, at least in part, by the separation of the generative AI model from the enterprise data of the enterprise information environment. The separation of the generative AI model from the enterprise data of the enterprise information environment may reduce hallucinations and information leakage of the generative AI model relative to other generative AI models that have been trained on other enterprise data from other enterprise information environments.

[0203] According to an example disclosed herein, a computer-implemented method is provided, comprising: processing a query input; identifying, based on the query input, one or more artificial intelligence applications and a plurality of associated data models from a plurality of data domains; analyzing the query input based on the data models from the plurality of data domains; determining, by a machine learning model, a relevance score based on the analysis of the query input; generating, based on the relevance score, one or more query sets for execution on at least one of the one or more artificial intelligence applications; and composing a response output based on the results of executing the one or more generated query sets.

[0204] The method may include generating a representation of the response output for display on an electronic display. The method may include displaying the representation of the response output on the electronic display. The method may include transmitting the response output to a computing device for display at the computing device.

[0205] The composition of the response output based on the results of executing one or more generated query sets depends on the technical functionality of one or more data models, one or more machine learning models and / or one or more artificial intelligence applications. One or more data models, one or more machine learning models and / or one or more artificial intelligence applications may have been trained using a machine learning algorithm. Thus, the operation of one or more data models, one or more machine learning models and / or one or more artificial intelligence applications can be based on parameters that have been learned through training (rather than parameters that have been set by a human programmer). One or more data models, one or more machine learning models and / or one or more artificial intelligence applications can be implemented in dedicated hardware. Additionally or alternatively, one or more data models, one or more machine learning models and / or one or more artificial intelligence applications can include simulating one or more data models, one or more machine learning models and / or one or more artificial intelligence applications in software.

[0206] One or more data models, one or more machine learning models, and / or one or more artificial intelligence applications may have been configured through training based on training data. The training data may be different from the data in multiple data domains. In other words, one or more data models, one or more machine learning models, and / or one or more artificial intelligence applications may not have been trained on data from multiple data domains. This can prevent information leakage from data stored in multiple data domains, thereby improving the security and integrity of enterprise datasets.

[0207] Processing the query input may include any of data modeling feature inspection and machine learning model simulation to select one or more analysis channels. The response output may include any of predictions, insights, and recommendations from the associated artificial intelligence application. The response output may include any of reports and dynamically configured dashboards based on any of the predictions, insights, and recommendations from the associated artificial intelligence application. One or more query sets for running on the associated artificial intelligence application can be configured to address a range of different artificial intelligence applications and datasets. The one or more query sets configured for a range of different artificial intelligence applications and datasets are based on a type system to define types for use by different artificial intelligence applications. The results of one or more generated query sets may involve multiple different artificial intelligence applications, and the results may include newly generated insights from multiple different artificial intelligence applications. Processing the query input may include natural language processing (NLP) with vectorized data, and may generate pre-trained transformers that are fine-tuned or retrained on specific data tailored to the type of query of the associated artificial intelligence application.

[0208] According to an example disclosed herein, a computer-implemented method is provided, comprising: receiving an enterprise search query, wherein the enterprise search query includes natural language input; retrieving a plurality of data records associated with at least a portion of enterprise data of an enterprise information environment based on the enterprise search query and one or more retriever models; determining, through the one or more retriever models, a corresponding relevance score for each of the retrieved data records; selecting at least one of the retrieved data records based on the corresponding relevance score; selecting at least a portion of at least one of the retrieved data records based on one or more enterprise access control protocols; determining, through one or more large language models, an enterprise access-controlled natural language output based on the selected at least a portion; and facilitating presentation of the enterprise access-controlled natural language output.

[0209] Facilitating presentation of the natural language output may include generating a representation of the natural language output for display on an electronic display. Facilitating presentation of the natural language output may include displaying the representation of the natural language output on an electronic display. Facilitating presentation of the natural language output may include transmitting the natural language output to a computing device for display at the computing device.

[0210] The one or more enterprise access control protocols may include a control protocol associated with credentials (e.g., user credentials and / or user profile) associated with input of a search query. For example, a search query may be entered at a computing device by a user who logs in or otherwise accesses a system or device using credentials, such as user credentials (e.g., login credentials). The credentials used when entering the query may have access controls associated with them. Access controls may include permissions or permissions associated with the credentials and may be used to restrict access to data from a data record. For example, access controls may be used to determine data from a data record that is authorized to be accessed and / or data from a data record that should not be authorized to be accessed. Selecting at least a portion of at least one of the retrieved data records based on the one or more enterprise access control protocols may include selecting unrestricted information based on the one or more enterprise access control protocols.

[0211] At least a portion of at least one of the retrieved data records is selected based on one or more enterprise access control protocols, and then one or more large language models are used to determine an enterprise access-controlled natural language output based on the selected portion of at least one of the retrieved data records. This can mean only providing the large language model with data that has been selected based on the access control protocol. The large language model can incorporate probabilistic methods such that portions of the large language model's output may be difficult to trace back to specific inputs. By limiting the input provided to the large language model to include only information selected based on the access protocol, the large language model does not generate output based on any data or information that the access control protocol does not permit. This can prevent the large language model's output from being inadvertently based on data or information that the access control protocol does not permit.

[0212] Access control protocols can be used to provide a technical effect of improving the security and integrity of data storage. This is different from other model-based (e.g., generative AI-based) response systems that may be subject to information leakage when information and data are ingested into the model and / or during the training of the model with no or limited traceability or security controls.

[0213] Utilizing one or more large language models to determine enterprise access controlled natural language output based on at least one selected portion may include providing the at least one selected portion to the large language model as input to the large language model. Selecting at least a portion of at least one of the retrieved data records based on one or more enterprise access control protocols may include restricting at least a portion of at least one of the retrieved data records to be included in the at least a portion of at least one of the retrieved data records. Within the scope of the present application, it is expressly intended that the various aspects, embodiments, examples and alternatives set forth in the preceding paragraphs, in the claims and / or in the specification and drawings, and in particular individual features thereof, may be employed independently or in any combination. That is, all examples and / or features of any example may be combined in any manner and / or combination unless such features are incompatible.

[0214] The execution of certain operations may be distributed among processors, not only residing within a single machine, but also deployed across multiple machines. In some example embodiments, the processor or processor-implemented modules may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented modules may be distributed across multiple geographic locations.

[0215] Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are shown and described as separate operations, one or more of the individual operations may be performed simultaneously and need not be performed in the order shown. Structures and functionality presented as separate components in the example configurations may be implemented as combined structures or components. Similarly, structures and functionality presented as single components may be implemented as separate components. Thus, for example, enterprise generative artificial intelligence systems 302 and 400 may be implemented as separate components. Figure 3 and Figure 4 In the examples of FIG, each is depicted as a single system, however, each can also be implemented as multiple systems. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter of this invention.

[0216] The present invention has been described above with reference to exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made and other embodiments may be used without departing from the broader scope of the present invention. Therefore, the present invention is intended to encompass these and other variations of the exemplary embodiments.

Claims

1. A method comprising: generating a set of potential responses to the prompt using one or more data models utilizing data from at least a plurality of data domains of an enterprise information environment including access control; determining verification data for the set of potential responses, wherein the verification data is from the plurality of data domains of an enterprise information environment; selecting a deterministic response from the set of potential responses based on the scoring of the verification data and taking into account that profile information associated with the prompt is restricted based on the access control; as well as The selected deterministic response is output together with verification data corresponding to the selected deterministic response.

2. The method according to claim 1, further comprising: A traceability analysis of the validation data is generated, the traceability analysis being indicative of any of documents, document fragments, and insights of at least a portion of one or more enterprise datasets.

3. The method according to claim 1, wherein The scoring of the validation data includes: Based on the one or more data models, a plurality of relevance scores associated with at least a portion of the verification data for each verification data for the set of potential responses is determined.

4. The method according to claim 1, in, The one or more data models include a plurality of models trained for different data domains among the plurality of data domains, Each data model represents the corresponding relationships and attributes of different data domains in multiple different data domains, and The corresponding relationships and attributes include any of data types, data formats, and industry-specific information.

5. The method according to claim 1, wherein The one or more data models include a multimodal model, wherein at least one data model is a large language model.

6. The method according to claim 1, wherein The access control enforces restrictions including at least one of administrative policy, security policy, profile permissions, and organizational controls.

7. The method according to claim 1, wherein The access control enables selection of different deterministic responses based on profiles having different access rights.

8. The method according to claim 1, wherein The access control enables outputting different authentication data based on configuration files having different access rights.

9. The method according to claim 1, wherein The selected deterministic response includes at least one of a prediction, an insight, and a recommendation from an artificial intelligence application.

10. The method according to claim 1, wherein The output includes at least one of: data visualization, automated controls and instructions, reports, and dynamically configured dashboards.

11. A method comprising: Receive inquiries; Based on the query, identifying one or more enterprise data sets, one or more artificial intelligence applications, and one or more data models from a plurality of different data domains of an enterprise information environment; determining a plurality of relevance scores associated with at least a portion of the one or more enterprise data sets based on data models from the plurality of different data domains; determining, by one or more generative artificial intelligence models, specific information from the plurality of different data domains of the enterprise information environment based on the plurality of relevance scores and one or more enterprise access control protocols; generating a natural language output based on specific information from one or more related data domains of the plurality of different data domains; as well as Facilitating presentation of the natural language output.

12. The method according to claim 11, wherein The query comprises a natural language query received through a graphical user interface.

13. The method according to claim 11, wherein The one or more enterprise datasets include any of documents, document fragments, and insights generated by the one or more artificial intelligence applications.

14. The method according to claim 13, wherein: Each of the relevance scores is associated with a corresponding portion of the one or more enterprise datasets, and wherein each of the relevance scores is determined relative to other portions of the one or more enterprise datasets.

15. The method according to claim 11, wherein Each data model of the one or more data models corresponds to a different data domain of the plurality of different data domains.

Images