Natural language task distributed orchestration using generative machine learning model

Abstract

Data retrieval distributed orchestration for a generative machine learning model may be performed. When a natural language request associated with the generative application to perform a natural language task is received, one or more data retrievers may be selected to access the associated data repository according to a retrieval configuration previously specified for the generative natural language application. The data may then be retrieved by the selected data retriever and used to generate a prompt for the generative machine learning model. The results of the generative machine learning model may then be used to provide a response to the natural language request to perform the natural language task.

Description

Background Technology

[0001] As organizations enhance their technological capabilities in creating, tracking, and retaining information, a variety of technologies have been developed for managing and storing the ever-growing volume of information. Different types of data can be stored across many different systems or services. When it is necessary to locate the required information, it may be necessary to examine the different systems or services storing the data in order to retrieve the relevant information. Attached Figure Description

[0002] Figure 1 A logical block diagram illustrating the distributed orchestration of a natural language task using a generative machine learning model, according to some embodiments, is shown.

[0003] Figure 2 This is a logical block diagram illustrating a network of providers offering natural language generative application services according to some embodiments, the natural language generative application services implementing distributed orchestration of natural language tasks using generative machine learning models and intent classification for retrieval-enhanced generation pipelines that use generative machine learning models to perform retrieval-enhanced generation for natural language tasks.

[0004] Figure 3 This is a logical block diagram illustrating the interaction of creating a natural language generation application at a natural language generation application service, according to some embodiments.

[0005] Figure 4 This is a logic block diagram illustrating the interaction of adding a data repository according to some embodiments.

[0006] Figure 5 This is a logical block diagram illustrating the interaction with a retrieval system for a generative natural language application service, according to some embodiments.

[0007] Figure 6 This is a logical block diagram illustrating a data orchestration workflow for processing natural language requests, according to some embodiments.

[0008] Figure 7 This is a logical block diagram illustrating conversation history management according to some embodiments.

[0009] Figure 8 This is a high-level flowchart illustrating various methods and techniques for implementing distributed orchestration of natural language tasks using generative machine learning models, based on some embodiments.

[0010] Figure 9 This is a high-level flowchart illustrating various methods and techniques for obtaining relevant dialogue history when performing natural language tasks using generative machine learning models, based on some embodiments.

[0011] Figure 10A logical block diagram illustrating intent classification for a retrieval enhancement generation pipeline for natural language tasks using a generative machine learning model, according to some embodiments, is shown.

[0012] Figure 11 This is a logical block diagram illustrating the creation of an application main storage area for enforcing data access control, according to some embodiments.

[0013] Figure 12 This is a high-level flowchart illustrating, according to some embodiments, various methods and techniques for implementing intent classification using generative machine learning models to perform retrieval enhancement generation pipelines for natural language tasks.

[0014] Figure 13 This is a high-level flowchart illustrating various methods and techniques for creating application main storage areas for enforcing data access control, based on some embodiments.

[0015] Figure 14 Example systems configured to implement the various methods, techniques and systems described herein are shown according to some embodiments.

[0016] Although embodiments have been described herein by way of example with respect to several examples and illustrative drawings, those skilled in the art will recognize that the embodiments are not limited to the described embodiments or drawings. It should be understood that the drawings and detailed description thereof are not intended to limit the embodiments to the specific forms disclosed, but rather are intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope defined by the appended claims. The headings used herein are for organizational purposes only and are not intended to limit the scope of the specification or claims. As used throughout this application, the word “may” is used in a permissible sense (i.e., meaning possible) rather than in a mandatory sense (i.e., meaning mandatory). Similarly, the words “include,” “including,” and “includes” mean including but not limited to.

[0017] It should also be understood that although the terms "first," "second," etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the invention, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact. Detailed Implementation

[0018] This paper describes various techniques for distributed orchestration of natural language tasks using generative machine learning models. Specifically, it describes various techniques for intent classification that use generative machine learning models to perform retrieval augmentation pipelines for natural language tasks. Generative machine learning models refer to machine learning techniques that model different types of data to perform various data generation tasks given prompts. For example, natural language generative machine learning models (such as Large Language Models (LLMs)) are a type of generative machine learning model that refers to machine learning techniques applied to model languages, which can include natural languages ​​(e.g., human speech) and machine-readable languages ​​(e.g., programming languages, scripts, code representations, etc.). For a generative machine learning model that models language, it can receive language prompts and generate corresponding programming language predictions (which may be referred to as code predictions or code suggestions).

[0019] Generative machine learning models generate language to perform various natural language processing tasks. This form of machine learning enables language processing capabilities to be widely applied to many different systems, services, or applications. More generally, machine learning is the discipline of training computer systems to recognize patterns by repeatedly exposing them to training data. In unsupervised learning, self-organizing algorithms learn previously unknown patterns in a dataset without any provided labels. In supervised learning, such training data includes inputs, which are labeled (automatically or by human annotators) with “benchmark truths” corresponding to the inputs and outputs. To evaluate / validate the performance of a trained model, a portion of the training dataset is typically set aside during training. The use of a trained model in production is often referred to as “inference,” during which the model receives new data not present in its training dataset and provides outputs based on its learned parameters. The training and validation process can be repeated periodically or intermittently by using new training data to optimize the previously learned parameters of the production model and deploying new production models for inference, in order to mitigate the decline in model accuracy over time.

[0020] For generative machine learning models, "inference" can be the output predicted by the model to satisfy a linguistic prompt (e.g., creating a summary of a draft financial plan). The prompt can be instructions and / or input text in one (or more) languages ​​(e.g., programming languages). Different generative machine learning models can be trained to handle different types of prompts. Some generative machine learning models can be generalized through a wide variety of topics and then fine-tuned for specific applications and subject domains. Fine-tuning refers to further training performed on a given machine learning model, which can be done by using additional training data to adapt the model's parameters to a specific knowledge domain or task. For example, an LLM can be trained to recognize patterns in text form and generate text predictions in many different scientific fields, literature, transcribed human conversations, and other academic disciplines, and then fine-tuned to optimize its ability to perform language tasks in a specific domain.

[0021] Retrieval augmentation is another technique used to adapt generative machine learning models for performing tasks specific to a use case by incorporating relevant data as part of their application. For example, various data retrieval techniques can be implemented to identify and provide relevant data information to enhance the performance of generative machine learning models. Challenges arise when considering the frequency and complexity of accessing different data sources or determining how to handle diverse natural language requests—including whether, when, and to what extent retrieval augmentation can be leveraged to perform tasks applicable to relevant data. For instance, data retrieval used to augment generative machine learning results can benefit from access to diverse data sources. However, to support access to different data sources, systems often require custom design. The resource costs and expertise required to implement custom data retrieval for augmentation often prevent many different developers from building generative machine learning-driven applications.

[0022] In various embodiments, distributed orchestration of natural language tasks using generative machine learning models provides techniques for allowing customized retrieval from diverse resources with different locations, formats, and requirements, in order to integrate data sources to enhance the performance of generative machine learning tasks. In this way, optimal data sources available for producing the best generative results become accessible, thereby influencing the generative machine learning model. Implementing these techniques can improve the performance of generative machine learning systems by optimally using computational resources (e.g., using source-specific data retrievers) and coordinating data retrieval to obtain access to data most relevant to the received natural language task, thereby improving the quality and accuracy of generated results (e.g., preventing illusions).

[0023] Figure 1A logical block diagram illustrating a distributed orchestration of natural language tasks using generative machine learning models, according to some embodiments, is shown. Generative machine learning service 110 can be used for natural language processing (such as natural language generative application service 210), and / or for users who support or utilize other generative machine learning techniques besides natural language processing. Natural language tasks 102 (e.g., questions, instructions, or a combination of both) for the application can be received.

[0024] Generative machine learning service 110 can implement data retrieval management 120, which can use retrieval configuration 132 specified for natural language generative applications 130 (e.g., as a means of creating the following about Figure 3 The application discussed and / or the following text about Figure 5 (As part of the data retrieval process described) determines and invokes 121 to select data retrieval 140. Because data retrieval can be specifically selected and configured (as indicated at 130), custom data retrieval features can be implemented to provide access to the optimal data in data repository 150 without requiring generative application creators to develop custom retrieval and enhancement pipelines for generative applications.

[0025] The selected data retrieval unit 140 can access various data repositories 150 to obtain relevant data 125 and provide that data 127 and 104 to the prompt generation unit 160. The prompt generation unit 160 can then generate a prompt based on the obtained data (e.g., various sections of different documents, including natural language documents and non-natural language data, such as numerical data or data formatted according to various storage technologies, such as timestamps in database data formats). The prompt 106 can then be provided to a generative machine learning model 170 (e.g., a large language model), which is trained to perform the requested natural language task and produce a result 108, which can then be provided.

[0026] Please note that the preceding description is a logical explanation and should not be construed as a limitation on the implementation. Different combinations or implementation schemes may be implemented in various embodiments.

[0027] This specification begins with a general description of a network of providers implementing generative natural language application services that support distributed orchestration of natural language tasks using generative machine learning models. It then discusses various examples of distributed orchestration of natural language tasks using generative machine learning models, which include different components or component arrangements that can be used as part of the implemented services. Next, it discusses various different methods and techniques for implementing distributed orchestration of natural language tasks using generative machine learning models, some of which are illustrated in the accompanying flowcharts. Finally, it provides a description of example computing systems on which various components, modules, systems, devices, and / or nodes can be implemented. Various examples are provided throughout this specification.

[0028] Figure 2 This is a logical block diagram illustrating a provider network providing natural language generative application services according to some embodiments. These services implement distributed orchestration of natural language tasks using generative machine learning models and intent classification for performing retrieval-enhanced generative pipelines for natural language tasks using generative machine learning models. In some embodiments, provider network 200 may be a private or closed system, or may be established by an entity such as a company or public sector organization to provide one or more services (e.g., various types of cloud-based storage) accessible via the Internet and / or other networks to client 270. Provider network 200 may be implemented in a single location, or may include numerous data centers hosting various resource pools, such as a collection of physical and / or virtualized computer servers, storage devices, network equipment, etc. (e.g., hereinafter referred to as...). Figure 14The described computing system 2000 requires these data centers to implement and distribute the infrastructure and services provided by the provider network 200. In some embodiments, the provider network 200 may implement various computing systems, platforms, resources, or services, such as natural language generation application service 210, computing services, database services 230 (e.g., relational or non-relational (NoSQL) database query engines, mapping reduction processing, data stream processing, and / or other large-scale data processing technologies), data storage services 240 (e.g., object storage services, block-based storage services, or data storage services that can store different types of data for centralized access), data streaming and / or event services, and other services (any other type of network-based service, which may include various other types of storage, processing, analysis, communication, event processing, visualization, and security services not shown), including other services 260 that provide or generate datasets for access by the natural language generation application service 210.

[0029] In various embodiments, Figure 2 The components shown can be implemented directly within computer hardware, as instructions that can be executed directly or indirectly by computer hardware (e.g., a microprocessor or computer system), or through a combination of these technologies. For example, Figure 2 The components can be implemented by a system comprising multiple compute nodes (or simply nodes), each compute node being similar to Figure 14 The computer system embodiments shown and described below are illustrated. In various embodiments, the functionality of a given system or service component (e.g., a component of the data storage service 230) may be implemented by a specific node or may be distributed across several nodes. In some embodiments, a given node may implement the functionality of more than one service system component (e.g., more than one data storage component).

[0030] In various embodiments, the Natural Language Generative Application Service 210 can provide scalable, serverless, and machine learning-driven services to create or support generative natural language applications using application-specific data, such as data stored in database service 230, data storage service 240, or other services 260. The Natural Language Generative Application Service 210 enables users (e.g., enterprise customers) to deploy generative AI-driven "experts" within minutes. For example, in a unified, intuitive experience driven by generative AI, users (e.g., enterprise employees or agents) can ask complex questions, receive comprehensive answers, and perform actions on their enterprise applications via applications that operate based on enterprise data.

[0031] Natural Language Generative Application Service 210 easily connects to various systems, services, and applications hosted both within and outside of Provider Network 200 (e.g., other Provider Network / public cloud services, or on-premises / privately hosted systems). Once connected, Natural Language Generative Application Service 210 allows users to ask complex questions and perform actions on these systems using natural language (e.g., human voice commands). For example, a sales agent can request the generative application to compare various credit card offers and recommend the card with the best travel points for their customers, and Natural Language Generative Application Service 210 will support features such as providing recommendations, the rationale for those recommendations, and the data source references upon which the recommendations are based. In some scenarios, users can use the generative application to create case summaries and add them to a Customer Relationship Management (CRM) system.

[0032] Natural Language Generative Application Service 210 can implement a security layer that checks user permissions to prevent unauthorized access to enterprise systems, ensuring that users can only view information and perform actions they are authorized to do. Natural Language Generative Application Service 210 implements a guardrail to prevent and avoid incorrect or erroneous statements or other generated results (sometimes called illusions) by limiting responses to the data within the enterprise, and establishes trust by providing citations and references to the sources used to generate the answers. Natural Language Generative Application Service 210 can provide an intuitive user interface that allows users to create and deploy enterprise-grade applications in minutes without requiring expertise in the field of generative machine learning.

[0033] For example, businesses are striving to provide the new generative, AI-driven experiences their users expect when interacting with enterprise systems. Users may need to switch between multiple fragmented systems (such as internal wikis, various data-sharing sites, communication sites, or messaging services) to find information because they cannot piece together comprehensive answers from the ideas contained in multiple pieces of content. Furthermore, users cannot ask exploratory follow-up questions or perform comparative analysis on the content to better understand it. When users need to take any further action, they must traverse multiple platforms (such as CRM systems, ticketing systems, and other enterprise applications) to do so.

[0034] Recent advances in generative AI, driven by machine learning models trained to generate content (called generative machine learning models, such as generative language models like large language models (LLMs)), have opened up possibilities for building intuitive, expert-like experiences. However, these generative models have limitations because they lack sufficient understanding of enterprise data and their knowledge is not up-to-date. Generative models can also create illusions, and end users cannot verify the authenticity of responses. Furthermore, enterprises need to ensure that users do not obtain answers from content they do not have access to. Enterprises may also need to build and deploy conversational applications for users. This makes it difficult to apply new generative AI technologies to enterprise use cases. The lack of a unified, intuitive experience for enterprises leads to poor knowledge sharing among users, low self-service rates, and overall company-wide productivity losses.

[0035] Regarding Natural Language Generative Application Service 210, enterprises (and other service users) leverage its various features to overcome the technical challenges inherent in how enterprises utilize generative AI. Natural Language Generative Application Service 210 enables enterprises to easily leverage the capabilities of AI technologies, including generative AI, to securely transform how their users interact with their enterprise applications. Natural Language Generative Application Service 210 breaks away from the traditional fragmented, multi-system navigation experience, providing a single, unified, expert-like experience. Using intuitive interface elements (e.g., a simple click-based management interface), application creators (e.g., enterprise) can synchronize with enterprise systems. Users of generative applications benefit from numerous capabilities: such as generating answers from multiple documents, retrieving answers from knowledge embedded in models, comparative analysis, content summarization, mathematical and reasoning, text generation, and the ability to perform actions on enterprise applications. Natural Language Generative Application Service 210 can support requests to find information and perform subsequent actions (e.g., "Please find a strategy option for this customer and attach a summary to the customer's notes in the CRM system"). Natural Language Generation Application Service 210 uses enterprise content to generate answers, minimizing illusions and providing up-to-date information. To ensure user trust and security, Natural Language Generation Application Service 210 incorporates human-like citations, references, and attachments to source documents into its responses. Natural Language Generation Application Service 210 manages enterprise access and Access Control List (ACL) permissions. When a user asks a question to Natural Language Generation Application Service 210, it analyzes data within the enterprise system and generates responses only from content the user has the right to access. Natural Language Generation Application Service 210 also provides pre-built conversational applications that can be easily deployed to end users within minutes, accelerating the speed at which application creators realize value. The unified, intuitive experience provided by Natural Language Generation Application Service 210 improves enterprise productivity and knowledge sharing, and enhances end-user self-service.

[0036] In various embodiments, application creators can deploy generative applications that can leverage the Natural Language Generative Application Service 210 within their enterprise within minutes. For example, creators can quickly connect their enterprise systems to the Natural Language Generative Application Service 210 via a console or other graphical user interface. The Natural Language Generative Application Service 210 provides a wide variety of built-in data connectors for different data sources to correlate these data sources as a data repository for the generative application, and supports data retrievers that look up relevant data (e.g., documents or other non-natural language data, such as image data, numerical data, audio or video data) to feed into generative machine learning models (e.g., LLM). The Natural Language Generative Application Service 210 also supports numerous actions of the enterprise systems, such as updating customer records in a database or creating work orders in a problem management system, allowing users to perform actions in these applications using natural language commands. Next, application creators can associate their generative applications with their identity providers (e.g., within or outside provider network 200). Finally, application creators can deploy pre-built conversational applications to their end users.

[0037] Natural Language Generative Application Service 210 can support interaction through generative applications created (and, in some embodiments, hosted by Natural Language Generative Application Service 210) to perform various tasks that can be specified in natural language requests. Features of Natural Language Generative Application Service 210 used to support these interactions may include question-and-answer for enterprise data. For example, Natural Language Generative Application Service 210 can process end-user questions and return generative responses using information from various secure enterprise data sources. Natural Language Generative Application Service 210 can continue the conversation with the user within the context of an active session or begin a new one. Natural Language Generative Application Service 210 will support question-and-answer for both structured and unstructured data sources. Application creators (e.g., enterprise administrators) can choose whether they want to restrict responses from enterprise content or leverage knowledge from generative models to answer queries.

[0038] Another example feature of the Natural Language Generation Application Service 210 for supporting interaction can be security. The Natural Language Generation Application Service 210 provides ACL support across private data (e.g., enterprise data) and application-level security for enterprise systems. The Natural Language Generation Application Service 210 can generate responses based solely on content that the end user has the right to access. The Natural Language Generation Application Service 210 can provide the end user with reference information and other summary information from the source (e.g., a document) used to generate the response, enabling the user to perform fact-checking. Subsequent actions suggested by the Natural Language Generation Application Service 210 will only be performed on applications that the user has the right to access (e.g., database systems, CRM systems, etc.).

[0039] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction can be actions. The Natural Language Generative Application Service 210 enables end users to perform actions on various applications (such as email, messaging, publishing, or other communication or data-sharing applications) using natural language commands. For example, an end user can request the Natural Language Generative Application Service 210 to update opportunities in a CRM system or create a work order in a work order system.

[0040] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction is summarization. End users can also request a summary of the content in their chat.

[0041] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction is its built-in data connectors. The Natural Language Generative Application Service 210 natively supports documents and other data retrievers for many different data storage systems, data search systems, database systems, or any other data repositories, including support for ACLs used on those systems. The connectors eliminate the heavy lifting involved in crawling data sources, extracting text content from files, and making that text content searchable.

[0042] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction could be usage analytics. The Natural Language Generative Application Service 210 allows application creators (e.g., administrators) to analyze end-user engagement metrics, including query counts, session counts, query counts per session, and popular queries. In this way, the application can be updated or modified based on usage analytics.

[0043] Another example feature of the natural language generative application service 210 for supporting interaction is personalization. The natural language generative application service 210 leverages the end user's context (such as role, location, etc.) and learns from past interactions (such as past searches and likes / dislikes feedback received by the user) to provide a personalized experience.

[0044] Natural Language Generative Application Service 210 can support various features for ingesting, indexing, and / or retrieving relevant data from associated data repositories used for generative applications. Features of Natural Language Generative Application Service 210 include the ability to correlate and ingest data from different data sources. Once the data sources are connected, Natural Language Generative Application Service 210 will process the data from these content sources and be ready to be deployed within minutes. However, if the application creator already has content in a retriever such as OpenSearch or other indexes, these retrievers can be easily integrated with Natural Language Generative Application Service 210.

[0045] As mentioned above, generative machine learning models can sometimes create seemingly good but actually incorrect or otherwise flawed answers, a phenomenon known as illusion. Additionally, generative machine learning models may pick up inappropriate content because these models are trained on large public datasets. These risks can undermine the accuracy and credibility of an application. Natural Language Generative Application Service 210 leverages several capabilities to address these issues. Natural Language Generative Application Service 210 combines generative machine learning models with application-specific data retrieval to provide question-answering functionality. Natural Language Generative Application Service 210 first uses a retrieval tool to find relevant data needed for the request from an associated data repository, then feeds a portion of the most relevant data into the generative machine learning model to obtain a synthetic response relevant to the content of the application creator (e.g., a business). Additionally, Natural Language Generative Application Service 210 provides citations and references to business documents used to generate the response, allowing end users to verify the accuracy of the answers. Natural Language Generative Application Service 210 also utilizes built-in prompt and response classifiers to detect inappropriate content, such as profanity, insults, and blasphemous language.

[0046] Natural Language Generative Application Service 210 provides application creators who want to integrate Natural Language Generative Application Service 210 with their own generative AI-driven applications with various interface elements and features, including APIs and UI components (e.g., code snippets or libraries that encapsulate the functionality of Natural Language Generative Application Service 210 but do not define a specific style of user interface). Using these APIs and headless components, application creators can embed features of Natural Language Generative Application Service 210 into their own applications.

[0047] Natural Language Generative Application Service 210 offers application creators a number of customization options, including but not limited to: (1) Optimize the response style, such as whether the answer is short or long, or whether the answer is generated or extracted.

[0048] (2) Configure "featured answers" for specific queries.

[0049] (3) Customize the natural language generation application service 210 to prioritize the results based on attributes such as content source, popularity, freshness and other content metadata.

[0050] (4) Create a custom lexicon to help the natural language generation application service 210 understand company-specific terms. For example, the natural language generation application service 210 can be trained to understand that MBP means mobile banking platform.

[0051] (5) Use custom document enhancement features during ingestion to enrich the content and make it more meaningful.

[0052] (6) It can add custom actions to internal applications so that the natural language generative application service 210 can perform the custom actions.

[0053] There may be scenarios where the natural language generation application service 210 cannot find or generate the expected result (e.g., an answer to a specific question). In such scenarios, the natural language generation application service 210 will respond that no answer was found and will return a list of documents or other data, which may contain information related to the question asked.

[0054] Natural Language Generation Application Service 210 supports various user interface creation methods, including programmatic interfaces, APIs, software development kits (SDKs), and / or graphical user interfaces (such as hosted web consoles). For example, the web console of Natural Language Generation Application Service 210 provides an easy way to get started. Application creators can point Natural Language Generation Application Service 210 to content sources and use experience builders to quickly deploy pre-built user interfaces to end users. As mentioned above, application creators can also apply customizations, such as response tuning, document enhancement customization, and synonym customization, to further improve answer accuracy. Natural Language Generation Application Service 210 can also integrate with unhosted applications using APIs.

[0055] The natural language capabilities of the natural language generation application service 210 enable it to understand any business or professional domain. However, for application-specific (e.g., enterprise-specific) vocabulary, application creators can use custom synonym features of the natural language generation application service 210 to tune it to recognize those words.

[0056] Natural Language Generation Application Service 210 can provide support for accessing various types of data files and formats, including but not limited to PDF, HTML, PowerPoint presentation files, word processing files, spreadsheet files, JavaScript Object Notation (JSON), comma-separated values ​​(CSV), rich text files (RTF), plain text, audio / video, images, and scanned documents. Natural Language Generation Application Service 210 can support multiple different human languages ​​for interactive purposes and to perform natural language tasks.

[0057] The Natural Language Generation Application Service 210 can securely store application data and use it only to provide services to the application's end users. Data can be encrypted using a key provided by the service or a key provided by the application creator.

[0058] In some embodiments, the Natural Language Generative Application Service 210 may implement a frontend 211. Frontend 211 may support various types of programmatic interfaces (e.g., Application Programming Interfaces (APIs)), command lines, and / or graphical user interfaces to support the management of datasets for analysis, requesting, configuring, and / or otherwise acquiring new or existing analyses and / or performing natural language queries, as described below. Frontend 211 may be a service that application creators (or application owners) will use to configure and build custom applications (e.g., for generative AI-driven dialogues). For example, in some embodiments, frontend 211 may support HTTPS / 2 for streaming use cases and fallback to HTTPS / 1.1 for non-streaming use cases. In some embodiments, frontend 211 may have browser support for the API and network socket support for the streaming interface. In various embodiments, frontend 211 may implement rate limiting, metering, authentication assurance, and authorization.

[0059] Frontend 211 can dispatch downstream service requests (and / or proxies) to natural language generative application services (e.g., control plane 212, natural language task orchestration 213, session storage 214, retrieval 215, ingestion and indexing 216, data access management 217, and application management 218). For example, frontend 211 can: dispatch requests to control plane 212 to set up top-level resources required by the generative application / account; dispatch requests to application management 218 to allow application configuration; dispatch requests to retrieval 215 to allow configuration of retrieval sources for the generative application; dispatch requests to session storage 214 to obtain session history (for the session history API); and dispatch requests to natural language task orchestration 213 to process generative requests.

[0060] In some embodiments, the Natural Language Generative Application Service 210 may implement a control plane 212. The control plane 212 may be a service that stores and manages a top-level account for a generative application (or multiple generative applications that can be created under a single account). The control plane 212 may also be a single point service for handling data protection regulations (e.g., GDPR), resource identification and tagging from other provider networks 200, and requests for operations such as deleting top-level resources. The control plane 212 may coordinate actions between other services of the Natural Language Generative Application Service 210, such as application management service 217 and retrieval service 215.

[0061] In some embodiments, the natural language generative application service 210 may implement ingestion and indexing 216. The ingestion and indexing service 216 may allow application creators to identify and index data to associate it as a data repository for generative applications. Ingestion and indexing 215 may index documents into a service index (e.g., via API calls). Ingestion and indexing 218 may be a service that stores documents in a service index for retrieval as part of performing a natural language task. In some embodiments, ingestion and indexing 2158 abstracts the underlying storage and types and may include model calls during indexing and retrieval operations. Model calls may be used to generate embedding vectors before data is indexed and may also be used to generate embedding vectors for data (e.g., query text) during retrieval calls.

[0062] In some embodiments, the natural language generative application service 210 may implement data access management 217. (See below for more information.) Figure 7 In detail, data access management 217 can create an application master store 750, which can utilize information obtained from data sources to generate mappings between different data sources and local user identities. These mappings can then be mapped to the end-user identities of the application. Similar techniques can be applied to groups. In this way, data access management 217 can provide or support access control to specific data in data repositories associated with the application, thereby restricting the data retrieved from these data repositories based on the data visible or available to the end-users of the generative application.

[0063] In some embodiments, the natural language generative application service 210 may implement application management 218. In various embodiments, natural language application management 218 may support the creation and hosting of generative applications that will be available to end users, for example, as a hosted service or applications published to endpoints, as described below. Figure 3This is discussed in detail. For example, application management 218 can implement static component distribution, network services that accept network requests (e.g., HTTP 1.1 communication protocol) for transmitting application data (e.g., conversation history, user identity, etc.), network socket services that provide browsers with bidirectional streaming and chat conversation capabilities, and metadata storage that allows applications to obtain runtime information (e.g., domain ID). Natural language generative application service 210 can support web browser generative applications and support direct authentication to external identity providers via Security Assertion Markup Language (SAML) Single Sign-On (SSO) protocol and / or other SSO protocols. Natural language generative application service 210 can be implemented such that hosted generative applications act as proxies for the front end 211 of natural language generative application service 210.

[0064] In some embodiments, the natural language generative application service 210 may implement natural language task orchestration 213. Natural language task orchestration 213 may execute workflows to perform natural language tasks received as natural language requests, as discussed above and below. Figure 8 This is discussed in detail. For example, natural language task orchestration can include various sub-components, systems, or microservices that, in addition to other operations, can acquire request input and information such as user IDs and filtering criteria, and process this request input and information through an orchestration process, which includes, but is not limited to, ensuring that query input does not contain profane language, obtaining dialogue context from session storage, rewriting and generating queries, retrieving one or more results from a retrieval service, sending information to a generative machine learning model, and sending information to a response classifier to ensure that the response is free of bias, profane language, and defamation.

[0065] In some embodiments, the natural language generative application service 210 may implement session storage 214. Session storage 214 may be responsible for ensuring that the context of the conversation is maintained (e.g., even if the socket connection is closed by the user). Session storage 214 may also provide data on the conversation history (as described below). Figure 6 (As discussed above). Session storage 214 can also provide data for analysis (e.g., the number of queries per session, the number of active sessions at a given time, etc., as described above). Session storage 214 can use session IDs and message IDs to track each conversation and the associated threads of that conversation associated with each user ID (which may be specific to a particular end user of the generative application, which may have multiple different users).

[0066] In some embodiments, the natural language generative application service 210 may implement retrieval 215. Retrieval service 215 may support data retrieval from retrieval sources, as described below. Figure 5 This is discussed in detail. For example, retrieval service 215 may implement metadata storage (e.g., retrieval configuration data 520 discussed below), which can be used to store all metadata associated with a particular retrieval. This metadata may be information related to access roles or other credentials, such as Identity and Access Management (IAM) roles, Virtual Private Network (VPN) information (for communicating with data sources in a VPN). Retrieval service 215 will retrieve data from the underlying retrieval source and associated data repository. In some embodiments, retrieval service 215 may have built-in integration with a data repository (e.g., a pre-built data retrieval tool), or it may support obtaining and applying information from the application creator to specify parameters / queries in order to build a data retrieval tool to retrieve data.

[0067] In various embodiments, database service 230 can be various types of data processing services that, in some embodiments, perform general or specialized data processing functions (e.g., analytics, big data queries, time-series data, graphical data, document data, relational data, structured data, or any other type of data processing operation) on data stored across multiple storage locations. For example, in at least some embodiments, database service 210 may include various types of database services (e.g., relational) for storing, querying, and updating data. Such services can be scalable and extensible enterprise-grade database systems. In some embodiments, queries may point to databases across database service 230 distributed across multiple physical resources, and the database system can be scaled up or down as needed. In different embodiments, the database system can work efficiently with various types and / or organizational database schemas. In some embodiments, clients / subscribers can submit queries or other requests (e.g., requests to add data) in various ways, such as interactively via an SQL interface to the database system or via an application programming interface (API). In other embodiments, external applications and programs can submit queries to the database system using Open Database Connectivity (ODBC) and / or Java Database Connectivity (JDBC) driver interfaces.

[0068] In some embodiments, database service 220 may be various types of data processing services for performing different functions (e.g., query or other processing engines for performing functions such as anomaly detection, machine learning, data lookup, or any other type of data processing operation). For example, in at least some embodiments, database service 230 may include a MapReduce service that creates a cluster of processing nodes that implement MapReduce functionality on data stored in one of the data storage services 240. Various other distributed processing architectures and technologies may be implemented through database service 230 (e.g., grid computing, sharding, distributed hashing, etc.). It should be noted that in some embodiments, data processing operations may be implemented as part of data storage service 230 (e.g., a query engine processing requests for specific data).

[0069] Data storage service 240 can implement different types of data storage for storing, accessing, and managing data on behalf of client 270 as a network-based service, enabling client 270 to operate the data storage system in a cloud or network computing environment. For example, in some embodiments, a data storage service 230 can be implemented as a centralized data storage, allowing other data storage services to access data stored in the centralized data storage for processing and / or storage within those other data storage services. Such data storage service 240 can be implemented as an object-based data storage and can provide storage and access to various object or file data storage for placing, updating, and obtaining data objects or files of various types, sizes, or series. Such data storage service 230 can be accessed via a programmatic interface (e.g., API) or a graphical user interface. Data storage service 240 can provide virtual block-based storage for maintaining data as part of data volumes that can be mounted or accessed similarly to local block-based storage devices (e.g., hard drives, solid-state drives, etc.) and can be accessed using block-based data storage protocols or interfaces (e.g., Internet Small Computer Interface (iSCSI)).

[0070] In various embodiments, data streaming and / or event services can provide resources to ingest, buffer, and process streaming data in real time, which may be a source of a data repository. In some embodiments, data streaming and / or event services may act as an event bus or other communication / notification for event-driven systems or services (e.g., events occurring on provider network 200 services and / or local systems or applications).

[0071] Generally, client 270 can encompass any type of client configurable to submit network-based requests to provider network 200 via network 280, including requests to materialized view management platform 210 (e.g., requests to create generative applications at a natural language generative application service). For example, client 270 may include a suitable version of a web browser, or may include plug-in modules or other types of code modules that can execute as an extension of or within an execution environment provided by the web browser. Alternatively, client 270 can encompass applications in provider network 200, such as generative applications (or their user interfaces), to implement various features, systems, or applications. (For example, using the Natural Language Generative Application Service 210 API to send natural language requests to perform various tasks (e.g., question answering, summarizing, or various other features discussed above). In some embodiments, such applications may include sufficient protocol support (e.g., for a suitable version of Hypertext Transfer Protocol (HTTP)) for generating and processing web-based service requests without necessarily implementing full browser support for all types of web-based data. That is, client 270 may be an application that can directly interact with provider network 200. In some embodiments, client 270 may generate web-based service requests based on a Representational State Transfer (REST) ​​style web-based service architecture, a document- or message-based web-based service architecture, or another suitable web-based service architecture.)

[0072] In some embodiments, client 270 may provide access to provider network 200 to other applications in a manner transparent to them. For example, client 270 may integrate with an operating system or file system to provide storage on one of the data storage services 240 (e.g., a block-based storage service). However, the operating system or file system may present different storage interfaces to applications, such as a conventional file system hierarchy of files, directories, and / or folders. In this embodiment, the storage system service model may be utilized without modifying the application. Instead, the details of interfacing with the data storage service 240 may be coordinated by client 270 and the operating system or file system on behalf of the application executing within the operating system environment.

[0073] Client 270 can transmit network-based service requests (e.g., natural language queries) to provider network 200 via network 280 and receive responses from provider network 200. In various embodiments, network 280 can encompass any suitable combination of networking hardware and protocols necessary to establish network-based communication between client 270 and provider network 200. For example, network 280 can typically encompass various telecommunications networks and service providers that jointly implement the Internet. Network 280 can also include private networks, such as local area networks (LANs) or wide area networks (WANs), and public or private wireless networks. For example, given client 270 and provider network 200, both may be provisioned within an enterprise with their own internal networks. In such embodiments, network 280 can include the hardware (e.g., modems, routers, switches, load balancers, proxy servers, etc.) and software (e.g., protocol stacks, accounting software, firewall / security software, etc.) necessary to establish network links between given client 270 and the Internet, and between the Internet and provider network 200. It should be noted that in some embodiments, client 270 may use a private network instead of the public Internet to communicate with provider network 200.

[0074] As described above, the natural language generative application service 210 can support communication with an external data source 290 via a network 280 in order to obtain data for performing various natural language tasks.

[0075] Figure 3This is a logical block diagram illustrating the interaction of creating a Natural Language Generation Application (NLP) at a Natural Language Generation Application Service, according to some embodiments. Application management 218 can support various requests for creating generational applications to perform natural language tasks using features of the Natural Language Generation Application Service 210. For example, application management 218 can support various features of the generational application to create web applications or other hosted applications. Even when the generational application itself is not hosted, unhosted applications can still be created to manage various backend features via requesting data, security, task orchestration, and other features for the generational application from frontend 211. Application management 218 can support the creation of generational applications, which can, for example, add any identity provider. The end user of the generational application should then be able to log in as the configured identity provider. In some embodiments, application management 218 can support the creation of custom headers on hosted generational applications (e.g., creating custom headers for web applications). Application management 218 can support adding custom prefixes to URLs or other web identifiers provided to access hosted generational applications. The created generative applications can support both managed and unmanaged applications, enabling chat / conversation interactions using the application's associated data repository and service-hosted generative machine learning models.

[0076] A request 302 to create an unmanaged application can be received. The creation request may include many of the aforementioned configuration features or parameters, such as identity providers, implementing or enabling various analytics collections, associating or specifying various associated data repositories, enabling / specifying various custom features (e.g., actions, styles, etc. as described above). Request processing 300 may be invoked by control plane 212 (which may be invoked by frontend 211, not shown) to execute the request and create configuration information for the unmanaged application 312 in application metadata 310. Various features of the unmanaged application can be changed in subsequent requests (not shown), such as adding or removing data repositories, adding, modifying, or removing custom features, or various other features of the unmanaged application. For the unmanaged application, application provisioning 320 may still assign an application identifier and / or other information, as indicated at 321. When the unmanaged generative language application 352 uses the provided identifier to invoke the Natural Language Generative Application Service 210 via frontend 211 to perform different tasks (e.g., in response to end-user interaction 354), as indicated at 356. Although not shown, an interaction with the identity provider may be performed prior to interaction 356 (e.g., direct interaction with the identity provider system / service via application 352). The end-user identity already determined by the identity provider (e.g., using login or other end-user identification procedures) may be included in information interaction 356 to be specific to the identified end-user.

[0077] In addition to adding managed application metadata 314, request processing 300 may also initiate application creation 305 for request 304 to create a hosted application. Application provisioning 320 may provision computing resources 330 and network endpoints for accessing the generative natural language application 332 (which can be configured according to various options supported by application management 218). For example, creation request 304 may include many of the aforementioned configuration features or parameters, such as identity providers, implementing or enabling various analytics collections, associating or specifying various associated data repositories, and enabling / specifying various custom features (e.g., actions, styles, etc. as described above). Various features of the unhosted application may be changed in subsequent requests (not shown), such as adding or removing data repositories, adding, modifying, or removing custom features, or various other features of the hosted application. Application provisioning 320 may obtain computing resources 330 (e.g., virtual computing resources used as host systems) from a computing service provider in provider network 200 and build the generative natural language application 332 according to the provided configuration features. For example, different software components corresponding to different selected features can be acquired and integrated based on application-specific information (e.g., identified data repositories, identified data retrievers, identity providers, etc.). The executable form of the generative application (e.g., compiled, assembled, or otherwise constructed) can then be installed as generative natural language application 332 on provisioned computing resources. A network endpoint (e.g., a network address, such as a URL) can be provided, allowing end users to access generative natural language application 332.

[0078] Once created, the generative natural language application 332 may be ready to accept end-user requests 344 and interact with the natural language generative application service 346 via frontend 211. An example interaction flow is described below. The end-user first accesses the network endpoint of the hosted generative application (e.g., a web application) and is redirected to a login page of a configured identity provider, where the end-user enters their username and password. After successful authentication, the end-user is directed to obtain access credentials for the generative natural language application 332 (e.g., using the SAMLRedirectAPI, where the identity provider provides a SAMLAssertion certificate, and then uses SAMLAssertion to call STS (Security Token Service) assumeRoleWithSAML to obtain sigv4 credentials (AccessKey, SecretKey)). The obtained credentials may be valid for a period of time (e.g., 1 hour) allowing the end-user access to the generative natural language application 332. The end-user is then directed to a homepage for final authentication and qualification (e.g., using cookies or other session-saved information). An authentication token can be obtained and used to establish connections for interactive features (e.g., network socket chat connections with frontend 211) and event streams by signing all calls with these credentials, and the connections and event streams can be stored in browser memory for further use until they expire.

[0079] Figure 4 This is a logical block diagram illustrating the interaction of adding a data repository according to some embodiments. A request 402 to add a repository and perform indexing can cause request processing 400 to initiate ingestion 410 to obtain 411 data from data source 401 and provide the ingested data 412 to index generation 420, which can generate an index based on a known pattern and store 409 the indexed data repository. Data repository metadata 430 can be updated 405 to add a new repository.

[0080] For example, ingestion 410 can implement different connectors (e.g., software components that interact with an agent or are deployed as agents) on data source 401. As described above, data source 401 can be various types of data storage sources, processing sources, message sending and receiving sources, streaming sources, or other information sources, either inside or outside provider network 200. Different connectors can implement different document interpreters, parsers, crawlers, or other features that can interpret and retrieve information from data source 401 for inclusion in the index. For example, ingestion 410 can extract metadata describing data objects (e.g., document-wide metadata describing authors, titles, publishers, etc.) and the data itself (e.g., as document text paragraphs). Once retrieved, the ingested data 412 can be provided to index generation 420 for index creation.

[0081] Index generation 420 can implement various indexing techniques to perform data retrieval when performing natural language tasks, as described below. Figure 5 and 6 As discussed above, indexes can support natural language search and can use fields, vectors, or other representations to model the underlying extracted data to support data retrieval by the data retriever. Different types of indexes can be implemented in different embodiments. For example, sparse indexes can be created that index data for specific fields, including those data objects (e.g., documents) that have said fields.

[0082] A 404 request to add a repository but not index it can be performed by updating the data repository metadata (405) (and may include schema information for searching / accessing the data repository). For example, the request may provide location information, such as a network address, access credentials, data format, or other schema information, to allow the data retriever to obtain data for the retrieval pipeline when performing natural language tasks, as described below.

[0083] Figure 5This is a logical block diagram illustrating the interaction with a retrieval system for a generative natural language application service, according to some embodiments. A retrieval request handler 510 (which may receive requests dispatched from front-end 211) may accept various retrieval requests 215. For example, a request to add a retriever 501, a request to remove a retriever 502, a request to update a retriever 503, a request to describe a retriever 504, or a request to list retrievers 505 may trigger a corresponding read / write 512 to the retriever configuration data 520. As part of adding and updating retrievers in the retriever configuration data 520, features such as the network location or endpoint of the data repository (e.g., indexed and unindexed repositories), authentication and authorization parameters, network configuration information (e.g., communicating with the data repository in a private network), access control information, retriever configuration (e.g., the number of items to retrieve), whether to paginate the results, data filtering, or various other data manipulations that can be performed on the retrieved data (e.g., reformatting the retrieved data).

[0084] Similar to the data connectors discussed above, different data retrievers 530 can implement corresponding interface components to generate data retrieval requests and send them to the appropriate data repository. For example, one type of data retriever 530 can read data from one type of index, and another type of data retriever 530 can read data from another type of index (or external data repository).

[0085] In some embodiments, data retrieval 530 may perform cross-retrieval comparisons to determine the relevance between data 534 obtained from different retrieval 530s, in some embodiments, to be provided in response 514. For example, cross-retrieval comparison 550 may transform the different data into a common format (e.g., using an ML encoder to encode the data into a common latent space for vector similarity comparison).

[0086] A request 506 invoking a selected retriever can cause a retrieval request handler 510 to dispatch a query 514 to a selected data retriever 530, which has access to data in a data repository 540. The schema information for some data repositories may be known (as it may have been generated during the ingestion of the indexed data repository). For other external or otherwise unindexed / uningested data repositories, schema information (which may be provided as part of an add retriever request 501) can be used to search for and return relevant data. See below for more information. Figure 6 As discussed, these data retrievers can be invoked by natural language task orchestration213.

[0087] Figure 6This is a logical block diagram illustrating a data orchestration workflow for processing natural language requests, according to some embodiments. As described above, natural language task orchestration 213 can interact with different services of generative natural language service 210 to perform natural language tasks. For example, session storage 214 can be accessed to obtain dialogue history information for a given natural language request, data access management 217 can be accessed to obtain specific data retrieval user information to enforce access control to the associated data repository, and retrieval 215 can be invoked to retrieve relevant data. The following description provides examples of task orchestration workflows that can be executed by natural language task orchestration 213 for each received task, as requested by the generative application.

[0088] Natural language requests for natural language tasks can be received, as indicated at 604. The task orchestration workflow 600 can implement a dialogue history 610. The dialogue history 610 can retrieve (if any) past dialogues for decontextualization. For example, a query / search can be performed on the session store 220 using a user identifier and / or session identifier to find other requests performed for the end user of the generative application. Multiple past sessions can be retrieved (if any). In some embodiments, the number can be determined from a large number of stored dialogues, rounds, or tasks (e.g., n most recent dialogues) based on a window of past dialogues, rounds, or other tasks. Dialogue data can be retrieved and provided for further processing. If no dialogue history exists, entries, data structures, or files can be created to store the dialogue history (including current natural language requests and tasks 602).

[0089] and Figure 1 The intent classification model 620, similar to the intent classification model 122 in the previous example, can be used to classify the intent of natural language requests, including tasks sent directly to prompt generation 640 and generative language model 650. In some embodiments, the intent classification model 620 can be a rule-based model that selects different intent classifications based on heuristics or other rules indicating different intents (e.g., looking for mathematical operators or conjunctions in the request to determine whether it is a multipart request, such as "add the earnings summary of X to the cash flow report of Y to generate a consolidated financial summary" or "if strategy type X is available in state Y, then use the information of Z to generate strategy type X").

[0090] In some embodiments, the intent classification model 620 can be implemented using machine learning-based methods that can be implemented for the intent classifier model 122. For example, neural network-based language models, such as Bidirectional Encoder Representations from Transformer (BERT) or Robustly Optimized BERT Pre-training Method (ROBERTA). These or various other machine learning models can be trained to recognize different intents. For example, the intent classifier model 620 can be trained to recognize communicative intents (without requiring the data retrieval pipeline 630) to detect general conversational natural language requests such as “Hello” or “How are you?”. For instructional or command intents including requests such as “Write an email,” “Summarize text,” “Write an article,” etc., the intent classifier model 620 can be further trained to detect instructional intents (including general commands and conversational commands). The intent classification model 620 can also be trained to recognize keyword requests (which may be queries that only input keywords without other context). For example, keyword requests may lack sufficient semantics and may be very short or overly technical. These keyword requests may not use a generative model (e.g., data retrieval might suffice), or they may require some query rewriting to become semantically meaningful. For example, searching for the IP address "172.1.2.100" or searching for specific terms like "MX-52113" (which could be a product number). Multipart tasks can also be trained similarly.

[0091] For some tasks, a single (or multi-part) task can be handled through the retrieval enhancement generation pipeline 630. The intent classification model 620 can classify tasks into retrieval tasks and non-retrieval tasks, where retrieval tasks are processed through the retrieval pipeline 630, and non-retrieval tasks are sent directly to the prompt generation 640. In some embodiments, multi-part tasks may include multiple (e.g., 0 to...) nRetrieval tasks and non-retrieval tasks. Non-retrieval tasks can include general conversational interactions (e.g., "casual conversation," such as "Hello," "Welcome to ABC," etc.) and tasks that can be performed without data retrieval (e.g., "Please divide 50,000 by 5,000"). Retrieval tasks can include instructions (e.g., "summarize," "describe," etc.), keywords (e.g., common entities in a data repository), and questions (sometimes called "queries"). If a conversation history is available, it can be provided to a generative language model (e.g., LLM) to rewrite instructions, keywords, or questions based on the conversation history (e.g., replacing ambiguous terms that can be determined from the conversation history, such as replacing pronouns with names or entities, adding additional terms such as "X's product or Y's service," etc.). Rewriting prompts can cause the generative machine learning model to return a rewritten form of the natural language request to perform the task (e.g., instruction or question), incorporating ambiguities or other clarifications arising from the conversation history. In some embodiments, the query rewriter 632 can be skipped in the absence of a conversation history.

[0092] The application's main storage area 636 can be used to provide the local user credentials or information to be used when retrieving data at data retrieval 634 (see below for details). Figure 7 The technique described allows mapping end-user service user identifiers of generative applications to local identifiers at various data repositories for ACL enforcement purposes. Data retrieval 634 can select (as indicated at 635) an appropriate data retriever (based on the application's configuration at creation or update time, as discussed above). Figure 3 (As discussed). Once the relevant data paragraphs have been obtained, they are provided to prompt generation 640.

[0093] In various embodiments, prompt generation 640 may implement a rule-based prompt generator that can generate prompts based on classification types (e.g., by completing corresponding prompt templates for each classification type), the prompts including a request and (if applicable) relevant data retrieved at pipeline 630 and a rewrite request at 632. A generative machine learning model 650 may be trained to generate a natural language response to the prompt generated at 540. In some embodiments, the generative machine learning model 650 may be an LLM, including a privately developed or maintained base model (FM) that can use millions or billions of parameters to generate a response to the prompt. As part of the prompt, a request may be included using the provided relevant data (retrieved via pipeline 630) such that the generative machine learning model 650 does not return a response generated by illusion. The generative machine learning model 650 may be hosted as part of a natural language generative application service or as a separate service of provider network 200. In some embodiments, generative application creation may support the selection of a specific generative machine learning model from multiple available models, including models hosted outside of provider network 200.

[0094] The results of the generative language model 650 can then be evaluated 660 to determine if they are complete (e.g., the last part of a multipart problem), or a validation check can be performed to determine if the results are valid (if invalid, an error or other fault indication can be sent)). For example, the natural language task orchestration 213 can track the number of parts of a task that have been completed and return to an earlier stage in the workflow 630 to perform additional stages (e.g., based on the output of the previous part, or not based on the previous output).

[0095] In some embodiments, the source of the retrieved data used to generate the results can be indicated (670). For example, as described above, the response can be annotated using comments or other indications from the retrieved documents (e.g., based on document-level metadata from which retrieved document paragraphs have been obtained). In some embodiments, an additional machine learning model trained to detect profane language or other inappropriate content can be invoked on the results to ensure that the results are not invalidated by inappropriate content. In some embodiments, a response 604 indicating that the question cannot be answered (e.g., due to inappropriate results or insufficient relevant data provided from the retrieval pipeline) can be sent. Otherwise, a response 604 can be sent based on a response generated by a generative machine learning model 650.

[0096] Session storage 214 can store information about different sessions (e.g., a session can refer to a set of interactions). Session storage 214 can store historical data about past natural language tasks to provide further context for the current natural language task, since generative machine learning models are typically stateless. For example, a first request might refer to "service A," and a second request in the session might refer to "it," also meaning "service A." Without historical data, interpreting "it" could lead to incorrect results.

[0097] In various embodiments, session storage 214 may store data generated by downstream services (including, for example, those provided by downstream services). Figure 6 The session storage 214 can store all relevant information used by the underlying generative model (described in the diagram). In some embodiments, the session storage 214 can store all requests and responses exchanged in a conversation / session. In addition to requests and responses, the session storage 214 can also store metadata, such as retrieval data (e.g., document paragraphs retrieved from different data repositories for each request in the conversation). The session storage 214 can also store and provide access to all historical conversations and messages therein (e.g., viewing). The session storage 214 can store this data and provide corresponding read functionality (e.g., read, search, or other access features). In this way, application creators (e.g., administrators) can view all this historical data (e.g., over a period of time). The session storage 214 can also expose analytics data, such as the number of conversations at a given time and the number of queries in each conversation.

[0098] Figure 7 This is a logical block diagram illustrating dialogue history management according to some embodiments, which may be implemented as part of session storage 214. Dialogue history management 700 may support various requests. When a dialogue task (e.g., a conversation) of a natural language application is completed, a request 710 to add a conversation may be received, which may update data storage 702 712 to add new conversation data 720n. Similar requests may be used to add various other information that can be maintained by session storage 214 in a similar manner. In some embodiments, the history structure may be a linked list, a graph, or other structure for maintaining an ordered dialogue history. Pointers to previous conversation data (e.g., to conversation data 720n-1) may be added. In some embodiments, session identifiers and message identifiers may be used to associate sessions, messages, and other related information.

[0099] When dialogue history is needed (e.g., for decontextualization and query rewriting as described above), a request for the relevant dialogue history is received at 730. Management at 700 can search the structure to obtain those dialogues (e.g., 720n-1, 720n-2, 720n-3, and 720n-4) within the history window at 734 (in some embodiments, this may be a sliding or moving window), thereby retrieving more recent history, as indicated by the arrow. The relevant dialogue can then be returned, as indicated at 736.

[0100] Although described and illustrated in the context of a network of providers implementing natural language generative application services. Figure 2-7 ,but Figure 2-7 The various components shown and described herein can be readily applied to other natural query language processing techniques, systems, or devices that assist in performing natural language queries on datasets. Therefore, Figure 2-7 This is not intended to limit other embodiments of a system that can implement natural language query processing. Figure 8 This is a high-level flowchart illustrating various methods and techniques for implementing distributed orchestration of natural language tasks using generative machine learning models, based on some embodiments.

[0101] Various systems and devices can implement the methods and techniques described below, either individually or collaboratively. For example, as mentioned above... Figure 2-7 The described natural language generative application service can implement various methods. Alternatively, combinations of different systems and devices can implement these methods. Therefore, the examples above and / or any other systems or devices described as performing the methods shown are not intended to limit other different components, modules, systems, or configurations of systems and devices.

[0102] As indicated at 810, in some embodiments, natural language requests can be received via an interface to a generative machine learning service, i.e., to perform natural language tasks for the generative natural language application using one or more data repositories associated with the generative natural language application. For example, a hosted or unhosted generative application may send a request to the interface of the generative machine learning service (e.g., via an API) to perform a natural language task. In some embodiments, the request may include an existing session (e.g., an existing or ongoing chat) or be identified by said existing session, which uses network communication features (e.g., tokens and / or cookies) and utilizes a bidirectional communication protocol.

[0103] As indicated at 820, in some embodiments, the generative machine learning service may select one or more data retrievers to retrieve data from one or more data repositories for performing natural language tasks, based at least in part on a retrieval configuration previously specified for the generative natural language application via an interface in a previous request. For example, an orchestration component (e.g., executing a workflow) Figure 6 The natural language task orchestrator 213 of the workflow 600 can access application and / or retrieval configuration information to determine which retrievals the generative application uses.

[0104] As indicated at 830, in some embodiments, data for performing natural language tasks can be obtained by invoking one or more selected data retrievers to access data at one or more data repositories. For example, as Figure 5 As described herein, a request can be dispatched to a selected data retriever, which can utilize parameters (e.g., the number of results, the amount of paragraphs or data to be retrieved for each result generation, or various other features as described above) to access (e.g., parse, interpret, search, or otherwise retrieve) results similar to or related to the natural language task (e.g., using sparse retrieval techniques). In some embodiments, further relevance analysis (including cross-retrieval analysis), such as dense re-ranking techniques, can be performed to provide the top k data items (e.g., document paragraphs) based on the relevance of each data item to the natural language request performing the natural language task.

[0105] As indicated at location 840, in some embodiments, the generative machine learning service may generate prompts, at least in part, based on data, for a generative machine learning model trained to perform a natural language task. For example, as described above regarding... Figure 6 The rule-based prompt generator discussed can: map data to fields to be included in a prompt template for a task; and then include instructions to generate a response using the provided data.

[0106] As indicated at 850, in some embodiments, a prompt may be submitted to a generative machine learning model to perform a natural language task. As indicated at 860, a response to the natural language request may be returned via an interface to the generative machine learning service. The response may be generated at least in part based on the result of the prompt received from the generative machine learning model. As described above, in some embodiments, other post-processing of the results may be performed, including source attribution, verification, appropriate response verification, and integrity checks of the task processing workflow.

[0107] Maintaining historical data, such as conversation history, can improve the performance of generative machine learning models. However, the capacity of generative machine learning models to absorb historical information may be limited. The techniques for acquiring and providing relevant historical data may also restrict the capacity of generative machine learning models. Figure 9 This is a high-level flowchart illustrating various methods and techniques for obtaining relevant dialogue history for performing natural language tasks using a generative machine learning model, according to some embodiments. As indicated at 910, in various embodiments, the dialogue history structure of the natural language generative application can be accessed. For example, search requests for end users, session identifiers, and / or dialogue identifiers can be performed. In some embodiments, if dialogue history data exists (e.g., the dialogue history structure is comment-free), as indicated at 920, a relevant history window for the natural language task requested for the generative natural language application can be determined. For example, links between historical data (e.g., messages, dialogues, etc.) can be traced back until the window boundary is reached. Then, as indicated at 930, in some embodiments, the dialogue for performing the natural language task within the relevant history window can be obtained.

[0108] If natural language requests are directly provided for data retrieval or prompts to generative machine learning models to obtain results, some natural language requests may suffer from poor performance. For example, different types of tasks (including instructions, questions, and conversational interactions) may each involve different processing, so it's crucial to understand the performance of natural language requests and tasks optimally within the context of data retrieval (if any) and generative machine learning response generation. Therefore, techniques that better determine the intent behind natural language requests performing natural language tasks can achieve better performance across different natural language tasks by intelligently identifying when different types of processing are optimal for different natural language tasks. Thus, implementing intent classification for augmented generative pipelines that use generative machine learning models to perform retrieval for natural language tasks can improve the performance of generative machine learning systems by optimally using computational resources when appropriate (e.g., not performing data retrieval when not needed), decontextualizing requests (e.g., adding relevant information), and identifying and performing multi-part tasks when needed (e.g., identifying and performing multiple parts by classifying tasks into multiple parts before providing a response).

[0109] Figure 10A logical block diagram illustrating intent classification for performing a retrieval enhancement generation pipeline for natural language tasks using a generative machine learning model, according to some embodiments, is shown. Generative machine learning service 1010 can be used with natural language generative application services (such as natural language generative application service 210), and / or supports other generative machine learning techniques besides natural language processing. Natural language tasks 1002 (e.g., questions, instructions, or a combination of both) can be received. Natural language tasks 1002 can be associated with a natural language generative application that can perform various natural language tasks for a user using an associated data repository 1050. In this way, natural language processing tasks can be performed in a relevant, secure, and verifiable manner, limited to data from data repository 1050, to prevent generative model illusions.

[0110] Generative machine learning service 1010 can implement natural language task orchestration 1020 to handle implementation workflows (such as those mentioned above). Figure 5 The example workflow shown is associated with the natural language task of application 1002. As part of the orchestration natural language task, orchestration 1020 may implement an intent classifier model 1022. The intent classifier model 1022 may be trained to evaluate natural language task requests to determine different orchestration paths, including how many iterations of the retrieval enhancement generation pipeline 1060 can be performed. For example, the intent classifier model 1022 may classify the task into a single part or multiple parts, and different subtasks to be completed, and then pass the natural language task (or subtask) to the retrieval enhancement generation pipeline 1060, as indicated at 1004.

[0111] In some embodiments, machine learning-based methods can be implemented for the intent classifier model 1022. For example, neural network-based language models, such as Bidirectional Encoder Representations from Transformer (BERT) or Robustly Optimized BERT Pre-training Method (ROBERTA), can be used. These or various other machine learning models can be trained to recognize different intents. For example, the intent classifier model 1022 can be trained to recognize communicative intents (without requiring the data retrieval pipeline 1060) to detect general conversational natural language requests such as “Hello” or “How are you?”. For instructional or command intents, including requests such as “Write an email,” “Summarize text,” “Write an article,” etc., the intent classifier model 1022 can be further trained to detect instructional intents (including general commands and conversational commands). The intent classification model 1022 can also be trained to recognize keyword requests (which may be queries that only input keywords without other context). For example, keyword requests may lack sufficient semantics and may be very short or overly technical. These keyword requests may not use generative models (e.g., data retrieval may be sufficient) or may require some query rewriting to acquire semantic meaning. For example, searching for specific terms like IP "172.1.2.100" or "MX-52113" (which could be a product number). Multi-part tasks can also be trained similarly.

[0112] Retrieval pipeline 1060 can access the associated data repository of application 1023 and use the acquired data 1025 to generate enhancement prompts 1062 for generative machine learning model 1070. The result 1072 can be provided to natural language task orchestration 1020, which can then determine whether further steps (e.g., classification based on intent), whether the result is valid, or determine other post-processing tasks. Some natural language tasks 1002 can be determined to have zero iterations through retrieval pipeline 1060 and can instead be sent as prompts 1006 to the generative machine learning model. The final result 1008 can be provided to the natural language generative application upon implementation.

[0113] As described above, in some embodiments, generative natural language services can help enforce access restrictions when accessing a data repository, so that end users do not gain access to information they are not authorized to access (e.g., because multiple different end users of a generative application may have different levels of access permissions). Figure 11 This is a logical block diagram illustrating the creation of an application main storage area for enforcing data access control, according to some embodiments.

[0114] Data access management 217 can provide master information to determine which data can be accessed when performing natural language tasks. A data crawler 1132 (which may be implemented as part of ingestion 410) can crawl data from data source 1120. As indicated at 1142, the data can be acquired, formatted, and stored in an indexed data repository 1140. Additionally, access control data 1144 (e.g., access control lists used by users to access individual documents) can be acquired by the data crawler 1132.

[0115] Identity crawler 1134 may be implemented (e.g., as part of data access management 217) and access data source 1120 to obtain local user information and local group information (which may be obtained or provided by data source 1120 through access to data source identity management 1110). Identity crawler 1134 may be used to create main storage 1150 for the application. Service user 1152 may be created as a service-level entity to aggregate different local users (for the same user) to the corresponding source 1153, thus acting as a globally federated user identity for the end users of the generative application. Group membership identity 1154 of storage user 1152 may also be provided. Storage group 1156 may also be provided, indicating group users 1157 in the group. In some embodiments, maintenance operations may be performed to update the application main data storage 1150 based on user changes (e.g., by listening for or receiving notifications of user changes from data source 1120).

[0116] The techniques discussed above can improve access control performance in various scenarios. For example, an application environment can use data repositories that can be accessed using different identity providers and multiple applications linked to those identity providers (which can act as data sources). In such scenarios, a user can exist across two identity providers, resulting in each different identity provider having a different identifier. Furthermore, each application can also have its own user identifier, which is local to the user of that application, thus creating multiple user identities and user-local connections. These connections can be captured in the application's main storage, allowing data retrieval to identify and use appropriate local user restrictions based on the service user's local user mapping 1153, group membership 1154, and information about access control data 1144 for our group users 1157, which is maintained and enforced for different indexed data repositories.

[0117] Although described and illustrated in the context of a network of providers implementing natural language generative application services. Figure 11 ,but Figure 11The various components shown and described herein can be readily applied to other natural query language processing technologies, systems, or devices that provide this functionality. Therefore, Figure 7 This is not intended to limit other embodiments of a system that can implement natural language query processing. Figure 12 This is a high-level flowchart illustrating, according to some embodiments, various methods and techniques for implementing intent classification using generative machine learning models to perform retrieval enhancement generation pipelines for natural language tasks.

[0118] As indicated at 1210, in some embodiments, natural language requests can be received via an interface to a generative machine learning service, i.e., to perform natural language tasks for the generative natural language application using one or more data repositories associated with the application. For example, a hosted or unhosted generative application may send a request to the interface of the generative machine learning service (e.g., via an API) to perform a natural language task. In some embodiments, the request may include an existing session (e.g., an existing or ongoing chat) or be identified by said existing session, which uses network communication features (e.g., tokens and / or cookies) and utilizes a bidirectional communication protocol.

[0119] As indicated at location 1220, in some embodiments, a classification machine learning model trained to determine the intent of a natural language request can be used to determine the intent of the natural language request. For example, as described above regarding Figure 1 The discussion suggests that machine learning models can be implemented using neural networks or other language-based classification models. These models can take the text of a natural language request as input, encode the text into representations or input features (e.g., feature vectors), and generate inferences that predict the intended classification (e.g., small talk, keywords, instructions / commands, queries / questions, single-part tasks, or multi-part tasks).

[0120] As indicated at 1230, in some embodiments, the number of iterations for the retrieval pipeline to perform the natural language task of the natural language request can be determined at least in part based on the intent of the natural language request. For example, different intents may not require data retrieval, while other intents (e.g., multipart, instruction, query / question, keyword) may require one or more retrieval iterations. For example, the orchestration system of the natural language task orchestrator 213 can be implemented as part of a generative machine learning service and can track progress and ensure performance based on the number of iterations determined by intent.

[0121] As indicated at location 1240, in some embodiments, natural language requests can be processed via a retrieval pipeline based on a determined number of iterations. (See above regarding...) Figure 6In detail, additional features can be implemented (in various combinations), such as query rewriting, conversation history, and end-user-specific access control information for generative applications, to retrieve different data from the data repository using a retriever.

[0122] Figure 13 This is a high-level flowchart illustrating various methods and techniques for creating an application main storage area for enforcing data access control, according to some embodiments. As indicated at 1310, in some embodiments, local user information and local group information can be obtained for a data source associated with a natural language generative application. For example, as described above regarding... Figure 7 As discussed, data crawlers and / or ingestion crawlers (or other software components) can search for and identify relevant access control information, as well as local user information and local group information.

[0123] As indicated at 1320, in some embodiments, a service-level main store can be created that maps service users to local users and local groups found in local user information and local group information. For example, service users can be viewed as a globally federated user map of end users of a generative application. In this way, the service-level main store can provide local user information to enforce access control across multiple different data repositories that may be involved in performing natural language tasks, as described above.

[0124] As indicated at 1330, in some embodiments, a service-level primary storage area may be provided for enforcing local access control at a data repository storing data from a data source when executing a natural language request from a natural language generative application. For example, as Figure 6 As indicated, the service-level primary storage can be invoked as part of a natural language task orchestration workflow to look up relevant local user / local group or other user information for enforcing access control information when performing data retrieval and / or preparing prompts for generative machine learning models to respond.

[0125] In various embodiments, the methods described herein can be implemented using any combination of hardware and software. For example, in one embodiment, the method can be implemented by a computer system (e.g., such as...). Figure 14 The computer system described herein includes one or more processors that execute program instructions stored on a computer-readable storage medium coupled to the processor. The program instructions can be configured to implement the functions described herein (e.g., the functions of various servers and other components of a network-based virtual computing resource provider described herein). Example embodiments of the methods are shown in the accompanying drawings and described herein. The order of any method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0126] The embodiments of distributed orchestration of natural language tasks using generative machine learning models described herein can be executed on one or more computer systems, which can interact with a variety of other devices. The embodiments of intent classification described herein for performing a retrieval augmentation generation pipeline for natural language tasks using generative machine learning models can also be executed on one or more computer systems, which can interact with a variety of other devices. Figure 14 One such computer system is illustrated. In various embodiments, the computer system 2000 can be any device of various types, including but not limited to personal computer systems, desktop computers, laptop computers, notebook or netbook computers, mainframe computers, handheld computers, workstations, network computers, cameras, set-top boxes, mobile devices, consumer devices, video game consoles, handheld video game devices, application servers, storage devices, peripheral devices such as switches, modems, routers, etc., or generally any type of computing device, computing node, computing node, computing system, computing system, or electronic device.

[0127] In the illustrated embodiment, computer system 2000 includes one or more processors 2010 coupled to system memory 2020 via input / output (I / O) interface 2030. Computer system 2000 also includes a network interface 2040 coupled to I / O interface 2030, and one or more input / output devices 2050, such as cursor control device 2060, keyboard 2070, and display 2080. Display 2080 may include a standard computer monitor and / or other display systems, technologies, or devices. In at least some embodiments, input / output device 2050 may also include a touch- or multi-touch-enabled device, such as a tablet or input pad, through which a user types input using a stylus device and / or one or more fingers. In some embodiments, it is contemplated that the embodiment may be implemented using a single instance of computer system 2000, while in other embodiments, multiple such systems or multiple nodes constituting computer system 2000 may host different portions or instances of the embodiment. For example, in one embodiment, some elements may be implemented via one or more nodes of computer system 2000 that are different from those implementing other elements.

[0128] In various embodiments, computer system 2000 may be a single-processor system including one processor 2010, or a multiprocessor system including several processors 2010 (e.g., two, four, eight, or another suitable number of processors). Processor 2010 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 2010 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs) such as x86, PowerPC, SPARC, or MIPS ISA or any other suitable ISA. In a multiprocessor system, each of the processors 2010 may typically, but does not necessarily, implement the same ISA.

[0129] In some embodiments, at least one processor 2010 may be a graphics processing unit (GPU). A GPU can be considered a dedicated graphics rendering device for a personal computer, workstation, game console, or other computing or electronic device. Modern GPUs can be highly efficient in manipulating and displaying computer graphics, and their highly parallel architecture makes them more efficient than a typical CPU for a range of complex graphics algorithms. For example, a GPU may implement graphics primitive operations in a manner that makes performing multiple graphics primitive operations much faster than drawing directly to the screen using a host central processing unit (CPU). In various embodiments, graphics rendering may be implemented at least in part by program instructions configured to execute on one of such GPUs or in parallel on two or more such GPUs. The GPU may implement one or more application programming interfaces (APIs) that allow programmers to invoke the GPU's functionality. Suitable GPUs are commercially available from vendors such as NVIDIA Corporation and ATI Technologies (AMD).

[0130] System memory 2020 may store program instructions and / or data accessible to processor 2010. In various embodiments, system memory 2020 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In the illustrated embodiment, program instructions and data implementing the desired functionality described above are shown stored in system memory 2020 as program instructions 2025 and data storage device 2035, respectively. In other embodiments, program instructions and / or data may be received, transmitted, or stored on different types of computer-readable media or on similar media separate from system memory 2020 or computer system 2000. Generally, non-transitory computer-readable storage media may include storage media or memory media, such as magnetic or optical media, for example, a disk or CD / DVD-ROM coupled to computer system 2000 via I / O interface 2030. Program instructions and data stored via a computer-readable medium can be transmitted via a transmission medium or signal (e.g., an electrical signal, an electromagnetic signal, or a digital signal), which can be transmitted via a communication medium such as a network and / or a wireless link, for example, via a network interface 2040.

[0131] In one embodiment, I / O interface 2030 can coordinate I / O traffic between processor 2010, system memory 2020, and any peripheral devices within the device, including network interface 2040, or other peripheral interfaces such as input / output device 2050. In some embodiments, I / O interface 2030 can perform any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., system memory 2020) into a format suitable for use by another component (e.g., processor 2010). In some embodiments, I / O interface 2030 may include support for devices attached via various types of peripheral buses, such as the Peripheral Component Interconnect (PCI) bus standard or variants of the Universal Serial Bus (USB) standard. In some embodiments, the functionality of I / O interface 2030 may be divided into two or more separate components, such as a northbridge and a southbridge. Additionally, in some embodiments, some or all of the functionality of I / O interface 2030, such as the interface for system memory 2020, may be directly incorporated into processor 2010.

[0132] Network interface 2040 allows data to be exchanged between computer system 2000 and other devices (such as other computer systems) attached to the network, or between nodes of computer system 2000. In various embodiments, network interface 2040 may support: communication via wired or wireless general data networks (such as any suitable type of Ethernet); communication via telecommunications / telephone networks (such as analog voice networks or digital fiber optic communication networks); communication via storage area networks (such as fiber optic SANs); or communication via any other suitable type of network and / or protocol.

[0133] In some embodiments, the input / output device 2050 may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or retrieving data by one or more computer systems 2000. Multiple input / output devices 2050 may be present in the computer system 2000 or distributed across different nodes of the computer system 2000. In some embodiments, similar input / output devices may be separate from the computer system 2000 and may interact with one or more nodes of the computer system 2000 via a wired or wireless connection (such as through network interface 2040).

[0134] like Figure 14 As shown, memory 2020 may include program instructions 2025, which can implement various methods and techniques as described herein, and data storage device 2035, including various data accessible by program instructions 2025. In one embodiment, program instructions 2025 may include software elements as described herein and as illustrated in the accompanying drawings. Data storage device 2035 may include data that can be used in an embodiment. In other embodiments, other or different software elements and data may be included.

[0135] Those skilled in the art will understand that Computer System 2000 is illustrative only and is not intended to limit the scope of the techniques described herein. Specifically, computer systems and apparatuses may include any combination of hardware or software capable of performing the indicated functions, including computers, personal computer systems, desktop computers, laptop computers, notebook or netbook computers, mainframe computers, handheld computers, workstations, network computers, cameras, set-top boxes, mobile devices, network devices, internet-connected appliances, PDAs, wireless mobile phones, pagers, consumer devices, video game consoles, handheld video game devices, application servers, storage devices, peripherals such as switches, modems, routers, etc., or generally any type of computing or electronic device. Computer System 2000 may also be connected to other devices not shown, or alternatively may operate as a standalone system. Furthermore, in some embodiments, the functionality provided by the illustrated components may be combined in fewer components or distributed across additional components. Similarly, in some embodiments, the functionality of some illustrated components may not be provided and / or other additional functions may be available.

[0136] Those skilled in the art will also understand that while various items are shown as being stored in memory or on a storage device during use, these items or portions thereof may be transferred between memory and other storage devices for memory management and data integrity purposes. Alternatively, in other embodiments, some or all of the software components may be executed in memory on another device and communicate with the illustrated computer system via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or portable article of manufacture for retrieval by a suitable driver, examples of which are described above. In some embodiments, instructions stored on a non-transitory computer-accessible medium separate from computer system 2000 may be transmitted to computer system 2000 via a transmission medium or signal (such as electrical, electromagnetic, or digital signals transmitted via communication media such as networks and / or wireless links). Various embodiments may also include receiving, transmitting, or storing instructions and / or data implemented according to the foregoing description on a computer-accessible medium. Therefore, the invention can be practiced with other computer system configurations.

[0137] It should be noted that any distributed system embodiment or any component thereof described herein can be implemented as one or more network services. For example, a leader node within a data warehouse system can present data storage services and / or database services to clients as network-based services. In some embodiments, the network-based service can be implemented by software and / or hardware systems designed to support interoperable machine-to-machine interactions on a network. The network-based service can have an interface described in a machine-processable format, such as a Network Services Description Language (WSDL). Other systems can interact with the network service in a manner specified by the description of the network-based service's interface. For example, the network-based service can define various operations that other systems can invoke, and can define specific application programming interfaces (APIs) that other systems can expect to follow when requesting various operations.

[0138] In various embodiments, a network-based service can be requested or invoked using a message that includes parameters and / or data associated with the network-based service request. Such messages can be formatted according to a specific markup language such as Extensible Markup Language (XML), and / or encapsulated using a protocol such as Simple Object Access Protocol (SOAP). To execute a network service request, a network-based service client can use an internet-based application-layer transport protocol such as Hypertext Transfer Protocol (HTTP) to assemble a message including the request and transmit the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the network service.

[0139] In some embodiments, web services may be implemented using expressive state transfer (“RESTful”) techniques instead of message-based techniques. For example, web services implemented according to RESTful techniques may be invoked via parameters included in HTTP methods such as PUT, GET, or DELETE, rather than being encapsulated in SOAP messages.

[0140] The accompanying drawings and various methods described herein represent example embodiments of the methods. The methods can be implemented in software, hardware, or a combination thereof. The order of the methods can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0141] Various modifications and alterations can be made, as will be apparent to those skilled in the art to which this disclosure pertains. The invention is intended to cover all such modifications and alterations; therefore, the above description is to be considered illustrative rather than restrictive.

[0142] Embodiments of this disclosure may be described in accordance with the following terms: Clause 1. A system comprising: Multiple computing devices, each including at least one processor and a memory, are configured to implement a natural language generation application service, wherein the natural language generation application service is configured to: Receive natural language requests via the interface of the natural language generative application service, that is, perform natural language tasks of the generative natural language application that are created at the natural language generative application service and use one or more data repositories associated with the generative natural language application. Access the retrieval configuration specified for the generative natural language application via the interface in the previous request; Based on the retrieval configuration, one or more data retrievers are selected to obtain data from the one or more data repositories for performing the natural language task; Invoke one or more selected data retrievers to retrieve the data from the one or more data repositories according to the natural language request; Based at least in part on the acquired data, prompts are generated for a generative machine learning model trained to perform the natural language task; The prompt is submitted to the generative machine learning model to perform the natural language task; The response to the natural language request is generated at least in part based on the results of the prompts received from the generative machine learning model; and The response to the request is returned via the interface.

[0143] Clause 2. The system according to Clause 1, wherein the natural language generative application service is configured to: Access the dialogue history structure of the generative natural language application; Determine the relevant history window for the natural language task; Retrieve one or more conversations within the relevant history window from the conversation history structure; and The natural language request is rewritten based on the one or more dialogues to decontextualize the natural language request, wherein the one or more data retrievers are invoked using the rewritten natural language request.

[0144] Clause 3. The system according to any one of Clauses 1 to 2, wherein the natural language generative application service is configured to: The interface receives a request to add one of the data retrievers to the generative natural language application; and Update the retrieval configuration to include the data retriever.

[0145] Clause 4. The system according to any one of Clauses 1 to 3, wherein the natural language generative application service is configured to: Receive a request to create the generative natural language application that will be hosted by the generative natural language application service; Pre-configure one or more computing resources to host the generative natural language application; and Provide a network endpoint for accessing the generative natural language application at the one or more computing resources, wherein the natural language request is submitted via the application programming interface of the generative natural language application.

[0146] Clause 5. A method comprising: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The generative machine learning service selects one or more data retrievers to retrieve data from the one or more data repositories for performing the natural language task, based at least in part on the retrieval configuration previously specified for the generative natural language application via the interface in a previous request; The generative machine learning service obtains the data for performing the natural language task by invoking one or more selected data retrievers to access the data at one or more data repositories in accordance with the natural language request; The generative machine learning service generates prompts for a generative machine learning model trained to perform the natural language task, based at least in part on the data. The generative machine learning service submits the prompts to the generative machine learning model to perform the natural language task; and The interface of the generative machine learning service returns a response to the natural language request, the response being at least in part based on the result of the prompt received from the generative machine learning model.

[0147] Clause 6. The method described pursuant to Clause 5 includes: The generative machine learning service accesses the dialogue history structure of the generative natural language application; The generative machine learning service determines the relevant historical record window for the natural language task; The generative machine learning service obtains one or more dialogues within the relevant history window from the dialogue history structure; and The generative machine learning service rewrites the natural language request based on the one or more dialogues to decontextualize the natural language request, wherein the one or more data retrieval machines are invoked using the rewritten natural language request.

[0148] Clause 7. The method according to any one of Clauses 5 to 6 further includes: Receive, via the interface, a request to add one of the one or more data retrievers to the generative natural language application; and The generative machine learning service updates the retrieval configuration to include the data retriever.

[0149] Clause 8. The method according to any one of Clauses 5 to 7 further includes: Receive a request via the interface to remove one of the data retrievers from the generative natural language application; and The generative machine learning service updates the retrieval configuration to remove one of the data retrievers.

[0150] Clause 9. The method according to any one of Clauses 5 to 8, wherein the generative machine learning service is implemented as part of a provider network, and wherein at least one of the data repositories is hosted outside the provider network.

[0151] Clause 10. The method according to any one of Clauses 5 to 9, wherein the retrieval configuration includes one or more parameters to be included in the access request from the one or more data retrievers to the one or more data repositories.

[0152] Clause 11. The method according to any one of Clauses 5 to 10, wherein at least one of the data repositories stores data as non-natural language data.

[0153] Clause 12. The method according to any one of Clauses 5 to 11, wherein at least one of the one or more data repositories is ingested and indexed by the generative machine learning service.

[0154] Clause 13. The method according to any one of Clauses 5 to 12 further includes: The generative machine learning service receives a request to create the generative natural language application to be hosted by the natural language generative application service; The generative machine learning service provisiones one or more computing resources to host the generative natural language application; and The generative machine learning service provides a network endpoint for accessing the generative natural language application at the one or more computing resources, wherein the natural language request is submitted via the application programming interface of the generative natural language application.

[0155] Clause 14. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The generative machine learning service selects one or more data retrievers to retrieve data from the one or more data repositories for performing the natural language task, based at least in part on the retrieval configuration previously specified for the generative natural language application via the interface in a previous request; The generative machine learning service obtains the data for performing the natural language task by invoking one or more selected data retrievers to access the data at one or more data repositories in accordance with the natural language request; The generative machine learning service generates prompts for a generative machine learning model trained to perform the natural language task, based at least in part on the data. The generative machine learning service submits the prompts to the generative machine learning model to perform the natural language task; and The interface of the generative machine learning service returns a response to the natural language request, the response being at least in part based on the result of the prompt received from the generative machine learning model.

[0156] Clause 15. One or more non-transitory computer-readable storage media as described in Clause 14, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: The generative machine learning service accesses the dialogue history structure of the generative natural language application; The generative machine learning service determines the relevant historical record window for the natural language task; The generative machine learning service obtains one or more dialogues within the relevant history window from the dialogue history structure; and The generative machine learning service rewrites the natural language request based on the one or more dialogues to decontextualize the natural language request, wherein the one or more data retrieval machines are invoked using the rewritten natural language request.

[0157] Clause 16. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 15, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: Receive, via the interface, a request to add one of the one or more data retrievers to the generative natural language application; and The generative machine learning service updates the retrieval configuration to include the data retriever.

[0158] Clause 17. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 16, wherein each of the one or more data retrievers interacts with different types of data storage systems to retrieve different portions of the data from different data repositories in the one or more data repositories.

[0159] Clause 18. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 17, wherein the retrieval configuration includes one or more parameters to be included in the access request from the one or more data retrievers to the one or more data repositories.

[0160] Clause 19. One or more non-transitory computer-readable storage media as described in Clauses 14 to 18, wherein at least one of the one or more data repositories is accessed by one of the one or more data retrievers using a mode provided as part of a request to add the one data repository.

[0161] Clause 20. One or more non-transitory computer-readable storage media as described in Clauses 14 to 19, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: The generative machine learning service receives a request to create a generative natural language application that is not hosted by the natural language generative application service; and The generative machine learning service provides an identifier for associating a request with the generative natural language application.

[0162] Clause 21. A system comprising: Multiple computing devices, each including at least one processor and a memory, are configured to implement a natural language generation application service, wherein the natural language generation application service is configured to: Receive natural language requests via the interface of the natural language generative application service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The intention of the natural language request is determined by a classification machine learning model trained to determine the intention of the natural language request. The number of iterations for the retrieval pipeline to execute the natural language task of the natural language request is determined at least in part based on the intent of the natural language request. The natural language request is processed through the retrieval pipeline according to a determined number of iterations, wherein the retrieval pipeline includes: At least one iteration of the said number of iterations, the natural language request for performing the natural language task is rewritten; At least one of the said iterations, data for performing the natural language task is retrieved from the one or more data repositories; Based at least in part on the retrieved data, prompts are generated for the generative machine learning model to perform the natural language task; The generative machine learning model is instructed to generate results according to the prompts; and The natural language request is returned via the interface of the natural language generative application service, the response being based at least in part on the results received from the generative machine learning model.

[0163] Clause 22. The system according to Clause 21, wherein the intent marks the natural language request as a non-retrieval instruction, wherein the determined number of iterations is zero, and wherein the natural language request is provided to the generative machine learning model to obtain the result.

[0164] Clause 23. The system according to any one of Clauses 21 to 22, wherein the natural language generative application service is configured to: The natural language generative application is used to obtain local user information and local group information from the data source; Create an application main storage area, which maps one or more local users found in the local user information to service users; and Provide the application's main storage area for enforcing access control at the one or more data access repositories associated with the natural language generative application.

[0165] Clause 24. The system according to any one of Clauses 21 to 23, wherein the natural language generative application service is configured to: Receive a request to create the generative natural language application that will be hosted by the generative natural language application service; Pre-configure one or more computing resources to host the generative natural language application; and Provide a network endpoint for accessing the generative natural language application at the one or more computing resources, wherein the natural language request is submitted via the application programming interface of the generative natural language application.

[0166] Clause 25. A method comprising: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The generative machine learning service enables a classification machine learning model trained to determine the intent of a natural language request to determine the intent of the natural language request. The generative machine learning service determines, at least in part, the number of iterations for the retrieval pipeline to perform the natural language task of the natural language request based on the intent of the natural language request; The generative machine learning service processes the natural language request through the retrieval pipeline according to a determined number of iterations, wherein the retrieval pipeline includes: At least one iteration of the said number of iterations, data for performing the natural language task is retrieved from the one or more data repositories; and The generative machine learning model is prompted to perform the natural language task, at least in part, based on the retrieved data; The interface of the generative machine learning service returns a response to the natural language request, the response being at least in part based on results received from the generative machine learning model.

[0167] Clause 26. The method according to Clause 25, wherein the intent is to mark the natural language request as a non-retrieval instruction, wherein the determined number of iterations is zero, and wherein the natural language request is provided to the generative machine learning model to obtain the result.

[0168] Clause 27. The method according to any one of Clauses 25 to 27 further includes obtaining the dialogue history of the generative natural language application, wherein the natural language request through the retrieval pipeline is processed at least in part based on the dialogue history.

[0169] Clause 28. The method according to any one of Clauses 25 to 27 further includes, in the result, one or more source attributions based on the one or more data repositories.

[0170] Clause 29. The method according to any one of Clauses 25 to 28, wherein the intention is to label the natural language task as comprising a plurality of subtasks, and wherein the determined number of iterations corresponds to two or more of the plurality of subtasks.

[0171] Clause 30. The method according to any one of Clauses 25 to 29 further includes first verifying the results of the generative machine learning model by the generative machine learning service before providing the results.

[0172] Clause 31. The method according to any one of Clauses 25 to 30 further includes: At the generative machine learning service, the natural language generative application receives local user information and local group information from the data source. The generative machine learning service creates an application main storage area, which maps one or more local users discovered in the local user information to service users; and The generative machine learning service provides the application's main storage area for enforcing access control at the one or more data access repositories associated with the natural language generative application.

[0173] Clause 32. The method according to Clause 31, wherein retrieving the data for performing the natural language task from the one or more data repositories requires access to the service user to obtain local user identity to access at least one of the one or more data repositories.

[0174] Clause 33. The method according to any one of Clauses 25 to 32 further includes: The generative machine learning service receives a request to create the generative natural language application to be hosted by the natural language generative application service; The generative machine learning service provisiones one or more computing resources to host the generative natural language application; and The generative machine learning service provides a network endpoint for accessing the generative natural language application at the one or more computing resources, wherein the natural language request is submitted via the application programming interface of the generative natural language application.

[0175] Clause 34. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform generative machine learning services: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The intention of the natural language request is determined by a classification machine learning model trained to determine the intention of the natural language request. The number of iterations for the retrieval pipeline to execute the natural language task of the natural language request is determined at least in part based on the intent of the natural language request. The natural language request is processed through the retrieval pipeline according to a determined number of iterations, wherein the retrieval pipeline includes: At least one iteration of the said number of iterations, data for performing the natural language task is retrieved from the one or more data repositories; and The generative machine learning model is prompted to perform the natural language task, at least in part, based on the retrieved data; The interface of the generative machine learning service returns a response to the natural language request, the response being at least in part based on results received from the generative machine learning model.

[0176] Clause 35. One or more non-transitory computer-readable storage media as described in Clause 34, wherein the intention is to mark the natural language request as non-instruction, wherein the determined number of iterations is zero, and wherein the natural language request is provided to the generative machine learning model to obtain the result.

[0177] Clause 36. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 35, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: obtaining the dialogue history of the generative natural language application, wherein the natural language request through the retrieval pipeline is processed at least in part based on the dialogue history.

[0178] Clause 37. One or more non-transitory computer-readable storage media pursuant to any one of Clauses 34 to 36, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: the result including attribution to one or more sources based on the one or more data repositories.

[0179] Clause 38. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 37, wherein the intention is to label the natural language task as comprising a plurality of subtasks, and wherein the determined number of iterations corresponds to two or more of the plurality of subtasks.

[0180] Clause 39. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 38, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: The natural language generative application is used to receive local user information and local group information from the data source; Create an application main storage area, which maps one or more local users found in the local user information to service users; and Provide the application's main storage area for enforcing access control at the one or more data access repositories associated with the natural language generative application.

[0181] Clause 40. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 39, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: The generative machine learning service receives a request to create a generative natural language application that is not hosted by the natural language generative application service; and The generative machine learning service provides an identifier for associating a request with the generative natural language application.

Claims

1. A system comprising: Multiple computing devices, each including at least one processor and memory, are configured to implement a generative machine learning service, wherein the generative machine learning service is configured to: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; One or more data retrievers are selected, at least in part, based on the retrieval configuration previously specified for the generative natural language application via the interface in a previous request, to retrieve data from the one or more data repositories for performing the natural language task; The data for performing the natural language task is obtained by invoking one or more selected data retrievers to access the data at one or more data repositories in accordance with the natural language request; Based at least in part on the data, prompts are generated for a generative machine learning model trained to perform the natural language task; The prompt is submitted to the generative machine learning model to perform the natural language task; and Return a response to the natural language request, the response being at least in part based on the result of the prompt received from the generative machine learning model.

2. The system according to claim 1, wherein the generative machine learning service is configured to: Access the dialogue history structure of the generative natural language application; Determine the relevant history window for the natural language task; Retrieve one or more conversations within the relevant history window from the conversation history structure; and The natural language request is rewritten based on the one or more dialogues to decontextualize the natural language request, wherein the one or more data retrievers are invoked using the rewritten natural language request.

3. The system according to claim 1 or claim 2, wherein the generative machine learning service is configured to: The interface receives a request to add one of the data retrievers to the generative natural language application; and Update the retrieval configuration to include the data retriever.

4. The system according to any one of claims 1 to 3, wherein the generative machine learning service is configured to: Receive a request to create the generative natural language application that will be hosted by the generative natural language application service; Pre-configure one or more computing resources to host the generative natural language application; and Provide a network endpoint for accessing the generative natural language application at the one or more computing resources, wherein the natural language request is submitted via the application programming interface of the generative natural language application.

5. A method comprising: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The generative machine learning service selects one or more data retrievers to retrieve data from the one or more data repositories for performing the natural language task, based at least in part on the retrieval configuration previously specified for the generative natural language application via the interface in a previous request; The generative machine learning service obtains the data for performing the natural language task by invoking one or more selected data retrievers to access the data at one or more data repositories in accordance with the natural language request; The generative machine learning service generates prompts for a generative machine learning model trained to perform the natural language task, based at least in part on the data. The generative machine learning service submits the prompts to the generative machine learning model to perform the natural language task; as well as The interface of the generative machine learning service returns a response to the natural language request, the response being at least in part based on the result of the prompt received from the generative machine learning model.

6. The method according to claim 5, further comprising: The generative machine learning service accesses the dialogue history structure of the generative natural language application; The generative machine learning service determines the relevant historical record window for the natural language task; The generative machine learning service obtains one or more dialogues within the relevant history window from the dialogue history structure; as well as The generative machine learning service rewrites the natural language request based on the one or more dialogues to decontextualize the natural language request, wherein the one or more data retrieval machines are invoked using the rewritten natural language request.

7. The method according to claim 5 or claim 6, further comprising: Receive a request via the interface to add one of the data retrievers to the generative natural language application; as well as The generative machine learning service updates the retrieval configuration to include the data retriever.

8. The method according to any one of claims 5 to 7, further comprising: Receive a request via the interface to remove one of the one or more data retrievers from the generative natural language application; as well as The generative machine learning service updates the retrieval configuration to remove one of the data retrievers.

9. The method according to any one of claims 5 to 8, wherein the generative machine learning service is implemented as part of a provider network, and wherein at least one of the data repositories is hosted outside the provider network.

10. The method of any one of claims 5 to 9, wherein the retrieval configuration includes one or more parameters to be included in the access request from the one or more data retrievers to the one or more data repositories.

11. The method according to any one of claims 5 to 10, wherein at least one of the data repositories stores data as non-natural language data.

12. The method according to any one of claims 5 to 11, wherein at least one of the one or more data repositories is ingested and indexed by the generative machine learning service.

13. The method according to any one of claims 5 to 12, further comprising: The generative machine learning service receives a request to create the generative natural language application to be hosted by the natural language generative application service; The generative machine learning service pre-configures one or more computing resources to host the generative natural language application; as well as The generative machine learning service provides a network endpoint for accessing the generative natural language application at the one or more computing resources, wherein the natural language request is submitted via the application programming interface of the generative natural language application.

14. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive natural language requests via the interface of the generative machine learning service, that is, use one or more data repositories associated with the generative natural language application to perform the natural language tasks of the generative natural language application; The generative machine learning service selects one or more data retrievers to retrieve data from the one or more data repositories for performing the natural language task, based at least in part on the retrieval configuration previously specified for the generative natural language application via the interface in a previous request; The generative machine learning service obtains the data for performing the natural language task by invoking one or more selected data retrievers to access the data at one or more data repositories in accordance with the natural language request; The generative machine learning service generates prompts for a generative machine learning model trained to perform the natural language task, based at least in part on the data. The generative machine learning service submits the prompts to the generative machine learning model to perform the natural language task; as well as The interface of the generative machine learning service returns a response to the natural language request, the response being at least in part based on the result of the prompt received from the generative machine learning model.

15. One or more non-transitory computer-readable storage media according to claim 14, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: The generative machine learning service accesses the dialogue history structure of the generative natural language application; The generative machine learning service determines the relevant historical record window for the natural language task; The generative machine learning service obtains one or more dialogues within the relevant history window from the dialogue history structure; and The generative machine learning service rewrites the natural language request based on the one or more dialogues to decontextualize the natural language request, wherein the one or more data retrieval machines are invoked using the rewritten natural language request.

Images