Segmented document indexing through data retrieval to enhance generative machine learning results

Abstract

An index is created using the segmented document to retrieve and enhance generation of responses to natural language requests using a generative machine learning model. When a natural language request is received, a search representation is generated and used to retrieve candidate portions of the document from the index. A relevance ranking is performed to identify a relevant portion of the document from the candidates and provide the relevant portion to prompt a generative machine learning model to provide a result of the natural language request.

Description

Background Technology

[0001] As organizations enhance their technological capabilities in creating, tracking, and retaining information, a variety of technologies have been developed for managing and storing the ever-growing volume of information. Different types of data can be stored across many different systems or services. When it is necessary to locate the required information, it may be necessary to examine the different systems or services storing the data in order to retrieve the relevant information. Attached Figure Description

[0002] Figure 1 A block diagram illustrating segmented document indexing that enhances generative machine learning results through data retrieval, according to some embodiments, is shown.

[0003] Figure 2 This is a logical block diagram illustrating a network of providers offering natural language generative application services according to some embodiments. These services implement segmented document indexing to enhance generative machine learning results through data retrieval and template-based tuning of generative machine learning models used to perform natural language tasks.

[0004] Figure 3 This is a logical block diagram illustrating the interaction of creating a natural language generation application at a natural language generation application service, according to some embodiments.

[0005] Figure 4 This is a logic block diagram illustrating the interaction of adding a data repository according to some embodiments.

[0006] Figure 5 This is a logical block diagram illustrating a data orchestration workflow for processing natural language requests, according to some embodiments.

[0007] Figure 6 This is a logical block diagram illustrating, according to some embodiments, how to use indexes of segmented documents to enhance data retrieval of generative machine learning results.

[0008] Figure 7 This is a high-level flowchart illustrating, according to some embodiments, various methods and techniques for implementing segmented document indexing that enhances generative machine learning results through data retrieval.

[0009] Figure 8A This is a high-level flowchart illustrating various methods and techniques for generating indexes of segmented documents, based on some embodiments.

[0010] Figure 8B This is a moving window shown according to some embodiments for document segmentation as part of index generation.

[0011] Figure 9A logic block diagram illustrating template-based tuning of a generative machine learning model for performing natural language tasks is shown according to some embodiments.

[0012] Figure 10 This is a logical block diagram illustrating the interaction with a retrieval system for a generative natural language application service, according to some embodiments.

[0013] Figure 11 This is a block diagram illustrating template-based tuning of a generative machine learning model for a generative application service, according to some embodiments.

[0014] Figure 12 This is a high-level flowchart illustrating, according to some embodiments, various methods and techniques for template-based tuning of generative machine learning models used to perform natural language tasks.

[0015] Figure 13 This is a high-level flowchart illustrating various methods and techniques for implementing template-based tuning of generative machine learning models, according to some embodiments.

[0016] Figure 14 Example systems configured to implement the various methods, techniques and systems described herein are shown according to some embodiments.

[0017] Although embodiments have been described herein by way of example with respect to several examples and illustrative drawings, those skilled in the art will recognize that the embodiments are not limited to the described embodiments or drawings. It should be understood that the drawings and detailed description thereof are not intended to limit the embodiments to the specific forms disclosed, but rather are intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope defined by the appended claims. The headings used herein are for organizational purposes only and are not intended to limit the scope of the specification or claims. As used throughout this application, the word “may” is used in a permissible sense (i.e., meaning possible) rather than in a mandatory sense (i.e., meaning mandatory). Similarly, the words “include,” “including,” and “includes” mean including but not limited to.

[0018] It should also be understood that although the terms "first," "second," etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the invention, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact. Detailed Implementation

[0019] This paper describes various techniques for segmented document indexing to enhance generative machine learning results through data retrieval. It also describes various template-based techniques for tuning generative machine learning models used to perform natural language tasks. Generative machine learning models refer to machine learning techniques that model different types of data to perform various data generation tasks given prompts. For example, natural language generative machine learning models (e.g., Large Language Models (LLMs)) are a type of generative machine learning model that refers to machine learning techniques applied to model languages, which can include natural languages ​​(e.g., human speech) and machine-readable languages ​​(e.g., programming languages, scripts, code representations, etc.). For generative machine learning models that model languages, the model can receive language prompts and generate corresponding programming language predictions (which may be referred to as code predictions or code suggestions).

[0020] Generative machine learning models generate language to perform various natural language processing tasks. This form of machine learning enables language processing capabilities to be widely applied to many different systems, services, or applications. More generally, machine learning is the discipline of training computer systems to recognize patterns by repeatedly exposing them to training data. In unsupervised learning, self-organizing algorithms learn previously unknown patterns in a dataset without any provided labels. In supervised learning, such training data includes inputs, which are labeled (automatically or by human annotators) with “benchmark truths” corresponding to the inputs and outputs. To evaluate / validate the performance of a trained model, a portion of the training dataset is typically set aside during training. The use of a trained model in production is often referred to as “inference,” during which the model receives new data not present in its training dataset and provides outputs based on its learned parameters. The training and validation process can be repeated periodically or intermittently by using new training data to optimize the previously learned parameters of the production model and deploying new production models for inference, in order to mitigate the decline in model accuracy over time.

[0021] For generative machine learning models, "inference" can be the output predicted by the model to satisfy a linguistic prompt (e.g., creating a summary of a draft financial plan). The prompt can be instructions and / or input text in one (or more) languages ​​(e.g., programming languages). Different generative machine learning models can be trained to handle different types of prompts. Some generative machine learning models can be generalized through a wide variety of topics and then fine-tuned for specific applications and subject domains. Fine-tuning refers to further training performed on a given machine learning model, which can be done by using additional training data to adapt the model's parameters to a specific knowledge domain or task. For example, an LLM can be trained to recognize patterns in text form and generate text predictions in many different scientific fields, literature, transcribed human conversations, and other academic disciplines, and then fine-tuned to optimize its ability to perform language tasks in a specific domain.

[0022] Retrieval-enhanced generation is another technique used to adapt generative machine learning models for performing tasks specific to particular use cases by incorporating the acquisition of relevant data as part of their application. For example, various data retrieval techniques can be implemented to identify and provide relevant data information in order to enhance the performance of generative machine learning models. Challenges arise when considering the frequency and complexity of accessing different data sources or determining how to handle different natural language requests—including whether, when, and to what extent retrieval-enhanced generation should be used to perform tasks applicable to relevant data. If less relevant data for performing natural language tasks is acquired and provided, some natural language requests may suffer from poor performance. Therefore, implementing segmented document indexing to enhance generative machine learning results through data retrieval can improve the performance of generative machine learning systems by optimally utilizing computational resources (e.g., by creating efficient and high-performance search indexes) and by providing appropriately sized relevant data to guide generative machine learning models to produce accurate results (e.g., preventing illusions).

[0023] Figure 1 A logical block diagram illustrating segmented document indexing that enhances generative machine learning results through data retrieval, according to some embodiments, is shown. The generative machine learning system 110 can be used for natural language processing (such as service 210), and / or supports other generative machine learning techniques in addition to natural language processing. It can receive natural language task requests 102 (e.g., questions, instructions, or a combination of both).

[0024] Generative machine learning system 110 can implement a retrieval enhancement pipeline or workflow to perform natural language requests 102. For example, data search 120 can implement sparse retrieval or other search techniques (e.g., dense retrieval) to access a data repository index 130, which includes data based on the information described below. Figure 4 , 8A The document portion 132 and document metadata 134 are segmented according to the techniques discussed in detail in 8B. The candidate portions obtained through the search can then be provided to a relevance rating 160, which can apply, for example, intensive re-grading (as described below). Figure 6 The techniques described above are used to rank the candidate parts. The selected candidate parts (based on rank) can then be used as part of prompt generation 160 (e.g., included as contextual input) to prompt the generative machine learning model 170 to generate the results of natural language request 102. The results generated by the generative machine learning model can be used to determine the response 104. In some embodiments, other post-result processing, such as verification, source attribution, and other techniques, can be performed.

[0025] Please note that the preceding description is a logical explanation and should not be construed as a limitation on the implementation. Different combinations or implementation schemes may be implemented in various embodiments.

[0026] This specification begins with a general description of a network of providers implementing generative natural language application services that support segmented document indexing that enhances generative machine learning results through data retrieval. Various examples of distributed orchestration of natural language tasks using generative machine learning models, comprising different components or component arrangements that can be used as part of the implemented services, are then discussed. Next, a variety of different methods and techniques for implementing segmented document indexing that enhances generative machine learning results through data retrieval are discussed, some of which are illustrated in the accompanying flowcharts. Finally, a description of example computational systems on which various components, modules, systems, devices, and / or nodes can be implemented is provided. Various examples are provided throughout this specification.

[0027] Figure 2This is a logical block diagram illustrating a provider network offering natural language generative application services according to some embodiments, the natural language generative application services implementing segmented document indexing to enhance generative machine learning results through data retrieval. In some embodiments, provider network 200 may be a private or closed system, or may be established by an entity such as a company or public sector organization to provide one or more services (e.g., various types of cloud-based storage) accessible via the Internet and / or other networks to client 270. Provider network 200 may be implemented in a single location, or may include numerous data centers hosting various resource pools, such as a collection of physical and / or virtualized computer servers, storage devices, network equipment, etc. (e.g., hereinafter referred to as...). Figure 14 The described computing system 1400 requires these data centers to implement and distribute the infrastructure and services provided by the provider network 200. In some embodiments, the provider network 200 may implement various computing systems, platforms, resources, or services, such as natural language generation application service 210, computing services, database services 230 (e.g., relational or non-relational (NoSQL) database query engines, mapping reduction processing, data stream processing, and / or other large-scale data processing technologies), data storage services 240 (e.g., object storage services, block-based storage services, or data storage services that can store different types of data for centralized access), data streaming and / or event services, and other services (any other type of network-based service (which may include various other types of storage, processing, analysis, communication, event processing, visualization, and security services not shown), including other services 260 that provide or generate datasets for access by the natural language generation application service 210.

[0028] In various embodiments, Figure 2 The components shown can be implemented directly within computer hardware, as instructions that can be executed directly or indirectly by computer hardware (e.g., a microprocessor or computer system), or through a combination of these technologies. For example, Figure 2 The components can be implemented by a system comprising multiple compute nodes (or simply nodes), each compute node being similar to Figure 14 The computer system embodiments shown and described below are illustrated. In various embodiments, the functionality of a given system or service component (e.g., a component of the data storage service 230) may be implemented by a specific node or may be distributed across several nodes. In some embodiments, a given node may implement the functionality of more than one service system component (e.g., more than one data storage component).

[0029] In various embodiments, the Natural Language Generative Application Service 210 can provide scalable, serverless, and machine learning-driven services to create or support generative natural language applications using application-specific data, such as data stored in database service 230, data storage service 240, or other services 260. The Natural Language Generative Application Service 210 enables users (e.g., enterprise customers) to deploy generative AI-driven "experts" within minutes. For example, in a unified, intuitive experience driven by generative AI, users (e.g., enterprise employees or agents) can ask complex questions, receive comprehensive answers, and perform actions on their enterprise applications via applications that operate based on enterprise data.

[0030] Natural Language Generative Application Service 210 easily connects to various systems, services, and applications hosted both within and outside of Provider Network 200 (e.g., other Provider Network / public cloud services, or on-premises / privately hosted systems). Once connected, Natural Language Generative Application Service 210 allows users to ask complex questions and perform actions on these systems using natural language (e.g., human voice commands). For example, a sales agent can request the generative application to compare various credit card offers and recommend the card with the best travel points for their customers, and Natural Language Generative Application Service 210 will support features such as providing recommendations, the rationale for those recommendations, and the data source references upon which the recommendations are based. In some scenarios, users can use the generative application to create case summaries and add them to a Customer Relationship Management (CRM) system.

[0031] Natural Language Generative Application Service 210 can implement a security layer that checks user permissions to prevent unauthorized access to enterprise systems, ensuring that users can only view information and perform actions they are authorized to do. Natural Language Generative Application Service 210 implements a guardrail to prevent and avoid incorrect or erroneous statements or other generated results (sometimes called illusions) by limiting responses to the data within the enterprise, and establishes trust by providing citations and references to the sources used to generate the answers. Natural Language Generative Application Service 210 can provide an intuitive user interface that allows users to create and deploy enterprise-grade applications in minutes without requiring expertise in the field of generative machine learning.

[0032] For example, businesses are striving to provide the new generative, AI-driven experiences their users expect when interacting with enterprise systems. Users may need to switch between multiple fragmented systems (such as internal wikis, various data-sharing sites, communication sites, or messaging services) to find information because they cannot piece together comprehensive answers from the ideas contained in multiple pieces of content. Furthermore, users cannot ask exploratory follow-up questions or perform comparative analysis on the content to better understand it. When users need to take any further action, they must traverse multiple platforms (such as CRM systems, ticketing systems, and other enterprise applications) to do so.

[0033] Recent advances in generative AI, driven by machine learning models trained to generate content (called generative machine learning models, such as generative language models like large language models (LLMs)), have opened up possibilities for building intuitive, expert-like experiences. However, these generative models have limitations because they lack sufficient understanding of enterprise data and their knowledge is not up-to-date. Generative models can also create illusions, and end users cannot verify the authenticity of responses. Furthermore, enterprises need to ensure that users do not obtain answers from content they do not have access to. Enterprises may also need to build and deploy conversational applications for users. This makes it difficult to apply new generative AI technologies to enterprise use cases. The lack of a unified, intuitive experience for enterprises leads to poor knowledge sharing among users, low self-service rates, and overall company-wide productivity losses.

[0034] Regarding Natural Language Generative Application Service 210, enterprises (and other service users) leverage its various features to overcome the technical challenges inherent in how enterprises utilize generative AI. Natural Language Generative Application Service 210 enables enterprises to easily leverage the capabilities of AI technologies, including generative AI, to securely transform how their users interact with their enterprise applications. Natural Language Generative Application Service 210 breaks away from the traditional fragmented, multi-system navigation experience, providing a single, unified, expert-like experience. Using intuitive interface elements (e.g., a simple click-based management interface), application creators (e.g., enterprise) can synchronize with enterprise systems. Users of generative applications benefit from numerous capabilities: such as generating answers from multiple documents, retrieving answers from knowledge embedded in models, comparative analysis, content summarization, mathematical and reasoning, text generation, and the ability to perform actions on enterprise applications. Natural Language Generative Application Service 210 can support requests to find information and perform subsequent actions (e.g., "Please find a strategy option for this customer and attach a summary to the customer's notes in the CRM system"). Natural Language Generation Application Service 210 uses enterprise content to generate answers, minimizing illusions and providing up-to-date information. To ensure user trust and security, Natural Language Generation Application Service 210 incorporates human-like citations, references, and attachments to source documents into its responses. Natural Language Generation Application Service 210 manages enterprise access and Access Control List (ACL) permissions. When a user asks a question to Natural Language Generation Application Service 210, it analyzes data within the enterprise system and generates responses only from content the user has the right to access. Natural Language Generation Application Service 210 also provides pre-built conversational applications that can be easily deployed to end users within minutes, accelerating the speed at which application creators realize value. The unified, intuitive experience provided by Natural Language Generation Application Service 210 improves enterprise productivity and knowledge sharing, and enhances end-user self-service.

[0035] In various embodiments, application creators can deploy generative applications that can leverage the Natural Language Generative Application Service 210 within their enterprise within minutes. For example, creators can quickly connect their enterprise systems to the Natural Language Generative Application Service 210 via a console or other graphical user interface. The Natural Language Generative Application Service 210 provides a wide variety of built-in data connectors for different data sources to correlate these data sources as a data repository for the generative application, and supports data retrievers that look up relevant data (e.g., documents or other non-natural language data, such as image data, numerical data, audio or video data) to feed into generative machine learning models (e.g., LLM). The Natural Language Generative Application Service 210 also supports numerous actions of the enterprise systems, such as updating customer records in a database or creating work orders in a problem management system, allowing users to perform actions in these applications using natural language commands. Next, application creators can associate their generative applications with their identity providers (e.g., within or outside provider network 200). Finally, application creators can deploy pre-built conversational applications to their end users.

[0036] Natural Language Generative Application Service 210 can support interaction through generative applications created (and, in some embodiments, hosted by Natural Language Generative Application Service 210) to perform various tasks that can be specified in natural language requests. Features of Natural Language Generative Application Service 210 used to support these interactions may include question-and-answer for enterprise data. For example, Natural Language Generative Application Service 210 can process end-user questions and return generative responses using information from various secure enterprise data sources. Natural Language Generative Application Service 210 can continue the conversation with the user within the context of an active session or begin a new one. Natural Language Generative Application Service 210 will support question-and-answer for both structured and unstructured data sources. Application creators (e.g., enterprise administrators) can choose whether they want to restrict responses from enterprise content or leverage knowledge from generative models to answer queries.

[0037] Another example feature of the Natural Language Generation Application Service 210 for supporting interaction can be security. The Natural Language Generation Application Service 210 provides ACL support across private data (e.g., enterprise data) and application-level security for enterprise systems. The Natural Language Generation Application Service 210 can generate responses based solely on content that the end user has the right to access. The Natural Language Generation Application Service 210 can provide the end user with reference information and other summary information from the source (e.g., a document) used to generate the response, enabling the user to perform fact-checking. Subsequent actions suggested by the Natural Language Generation Application Service 210 will only be performed on applications that the user has the right to access (e.g., database systems, CRM systems, etc.).

[0038] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction can be actions. The Natural Language Generative Application Service 210 enables end users to perform actions on various applications (such as email, messaging, publishing, or other communication or data-sharing applications) using natural language commands. For example, an end user can request the Natural Language Generative Application Service 210 to update opportunities in a CRM system or create a work order in a work order system.

[0039] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction is summarization. End users can also request a summary of the content in their chat.

[0040] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction is its built-in data connectors. The Natural Language Generative Application Service 210 natively supports documents and other data retrievers for many different data storage systems, data search systems, database systems, or any other data repositories, including support for ACLs used on those systems. The connectors eliminate the heavy lifting involved in crawling data sources, extracting text content from files, and making that text content searchable.

[0041] Another example feature of the Natural Language Generative Application Service 210 for supporting interaction could be usage analytics. The Natural Language Generative Application Service 210 allows application creators (e.g., administrators) to analyze end-user engagement metrics, including query counts, session counts, query counts per session, and popular queries. In this way, the application can be updated or modified based on usage analytics.

[0042] Another example feature of the natural language generative application service 210 for supporting interaction is personalization. The natural language generative application service 210 leverages the end user's context (such as role, location, etc.) and learns from past interactions (such as past searches and likes / dislikes feedback received by the user) to provide a personalized experience.

[0043] Natural Language Generative Application Service 210 can support various features for ingesting, indexing, and / or retrieving relevant data from associated data repositories used for generative applications. Features of Natural Language Generative Application Service 210 include the ability to correlate and ingest data from different data sources. Once the data sources are connected, Natural Language Generative Application Service 210 will process the data from these content sources and be ready to be deployed within minutes. However, if the application creator already has content in a retriever such as OpenSearch or other indexes, these retrievers can be easily integrated with Natural Language Generative Application Service 210.

[0044] As mentioned above, generative machine learning models can sometimes create seemingly good but actually incorrect or otherwise flawed answers, a phenomenon known as illusion. Additionally, generative machine learning models may pick up inappropriate content because these models are trained on large public datasets. These risks can undermine the accuracy and credibility of an application. Natural Language Generative Application Service 210 leverages several capabilities to address these issues. Natural Language Generative Application Service 210 combines generative machine learning models with application-specific data retrieval to provide question-answering functionality. Natural Language Generative Application Service 210 first uses a retrieval tool to find relevant data needed for the request from an associated data repository, then feeds a portion of the most relevant data into the generative machine learning model to obtain a synthetic response relevant to the content of the application creator (e.g., a business). Additionally, Natural Language Generative Application Service 210 provides citations and references to business documents used to generate the response, allowing end users to verify the accuracy of the answers. Natural Language Generative Application Service 210 also utilizes built-in prompt and response classifiers to detect inappropriate content, such as profanity, insults, and blasphemous language.

[0045] Natural Language Generative Application Service 210 provides application creators who want to integrate Natural Language Generative Application Service 210 with their own generative AI-driven applications with various interface elements and features, including APIs and UI components (e.g., code snippets or libraries that encapsulate the functionality of Natural Language Generative Application Service 210 but do not define a specific style of user interface). Using these APIs and headless components, application creators can embed features of Natural Language Generative Application Service 210 into their own applications.

[0046] Natural Language Generative Application Service 210 offers application creators a number of customization options, including but not limited to: (1) Optimize the response style, such as whether the answer is short or long, or whether the answer is generated or extracted.

[0047] (2) Configure "featured answers" for specific queries.

[0048] (3) Customize the natural language generation application service 210 to prioritize the results based on attributes such as content source, popularity, freshness and other content metadata.

[0049] (4) Create a custom lexicon to help the natural language generation application service 210 understand company-specific terms. For example, the natural language generation application service 210 can be trained to understand that MBP means mobile banking platform.

[0050] (5) Use custom document enhancement features during ingestion to enrich the content and make it more meaningful.

[0051] (6) It can add custom actions to internal applications so that the natural language generative application service 210 can perform the custom actions.

[0052] There may be scenarios where the natural language generation application service 210 cannot find or generate the expected result (e.g., an answer to a specific question). In such scenarios, the natural language generation application service 210 will respond that no answer was found and will return a list of documents or other data, which may contain information related to the question asked.

[0053] Natural Language Generation Application Service 210 supports various user interface creation methods, including programmatic interfaces, APIs, software development kits (SDKs), and / or graphical user interfaces (such as hosted web consoles). For example, the web console of Natural Language Generation Application Service 210 provides an easy way to get started. Application creators can point Natural Language Generation Application Service 210 to content sources and use experience builders to quickly deploy pre-built user interfaces to end users. As mentioned above, application creators can also apply customizations, such as response tuning, document enhancement customization, and synonym customization, to further improve answer accuracy. Natural Language Generation Application Service 210 can also integrate with unhosted applications using APIs.

[0054] The natural language capabilities of the natural language generation application service 210 enable it to understand any business or professional domain. However, for application-specific (e.g., enterprise-specific) vocabulary, application creators can use custom synonym features of the natural language generation application service 210 to tune it to recognize those words.

[0055] Natural Language Generation Application Service 210 can provide support for accessing various types of data files and formats, including but not limited to PDF, HTML, PowerPoint presentation files, word processing files, spreadsheet files, JavaScript Object Notation (JSON), comma-separated values ​​(CSV), rich text files (RTF), plain text, audio / video, images, and scanned documents. Natural Language Generation Application Service 210 can support multiple different human languages ​​for interactive purposes and to perform natural language tasks.

[0056] The Natural Language Generation Application Service 210 can securely store application data and use it only to provide services to the application's end users. Data can be encrypted using a key provided by the service or a key provided by the application creator.

[0057] In some embodiments, the Natural Language Generative Application Service 210 may implement a frontend 211. Frontend 211 may support various types of programmatic interfaces (e.g., Application Programming Interfaces (APIs)), command lines, and / or graphical user interfaces to support the management of datasets for analysis, requesting, configuring, and / or otherwise acquiring new or existing analyses and / or performing natural language queries, as described below. Frontend 211 may be a service that application creators (or application owners) will use to configure and build custom applications (e.g., for generative AI-driven dialogues). For example, in some embodiments, frontend 211 may support HTTPS / 2 for streaming use cases and fallback to HTTPS / 1.1 for non-streaming use cases. In some embodiments, frontend 211 may have browser support for the API and network socket support for the streaming interface. In various embodiments, frontend 211 may implement rate limiting, metering, authentication assurance, and authorization.

[0058] Frontend 211 can dispatch downstream service requests (and / or proxies) to natural language generative application services (e.g., control plane 212, natural language task orchestration 213, session storage 214, retrieval 215, ingestion and indexing 216, data access management 217, and application management 218). For example, frontend 211 can: dispatch requests to control plane 212 to set up top-level resources required by the generative application / account; dispatch requests to application management 218 to allow application configuration; dispatch requests to retrieval 215 to allow configuration of retrieval sources for the generative application; dispatch requests to session storage 214 to obtain session history (for the session history API); and dispatch requests to natural language task orchestration 213 to process generative requests.

[0059] In some embodiments, the Natural Language Generative Application Service 210 may implement a control plane 212. The control plane 212 may be a service that stores and manages a top-level account for a generative application (or multiple generative applications that can be created under a single account). The control plane 212 may also be a single point service for handling data protection regulations (e.g., GDPR), resource identification and tagging from other provider networks 200, and requests for operations such as deleting top-level resources. The control plane 212 may coordinate actions between other services of the Natural Language Generative Application Service 210, such as application management service 217 and retrieval service 215.

[0060] In some embodiments, the natural language generative application service 210 may implement ingestion and indexing 216. The ingestion and indexing service 216 may allow application creators to identify and index data to associate it as a data repository for generative applications. Ingestion and indexing 215 may index documents into a service index (e.g., via API calls). Ingestion and indexing 218 may be a service that stores documents in a service index for retrieval as part of performing a natural language task. In some embodiments, ingestion and indexing 2158 abstracts the underlying storage and types and may include model calls during indexing and retrieval operations. Model calls may be used to generate embedding vectors before data is indexed and may also be used to generate embedding vectors for data (e.g., query text) during retrieval calls.

[0061] In some embodiments, the natural language generative application service 210 may implement data access management 217. (See below for more information.) Figure 7 In detail, data access management 217 can create an application master store 750, which can utilize information obtained from data sources to generate mappings between different data sources and local user identities. These mappings can then be mapped to the end-user identities of the application. Similar techniques can be applied to groups. In this way, data access management 217 can provide or support access control to specific data in data repositories associated with the application, thereby restricting the data retrieved from these data repositories based on the data visible or available to the end-users of the generative application.

[0062] In some embodiments, the natural language generative application service 210 may implement application management 218. In various embodiments, natural language application management 218 may support the creation and hosting of generative applications that will be available to end users, for example, as a hosted service or an application published to an endpoint, as described below. Figure 3This is discussed in detail. For example, application management 218 can implement static component distribution, network services that accept network requests (e.g., HTTP 1.1 communication protocol) for transmitting application data (e.g., conversation history, user identity, etc.), network socket services that provide browsers with bidirectional streaming and chat conversation capabilities, and metadata storage that allows applications to obtain runtime information (e.g., domain ID). Natural language generative application service 210 can support web browser generative applications and support direct authentication to external identity providers via Security Assertion Markup Language (SAML) Single Sign-On (SSO) protocol and / or other SSO protocols. Natural language generative application service 210 can be implemented such that hosted generative applications act as proxies for the front end 211 of natural language generative application service 210.

[0063] In some embodiments, the natural language generative application service 210 may implement natural language task orchestration 213. Natural language task orchestration 213 may execute workflows to perform natural language tasks received as natural language requests, as discussed above and below. Figure 8A and 8B This is discussed in detail. For example, natural language task orchestration can include various sub-components, systems, or microservices that, in addition to other operations, can acquire request input and information such as user IDs and filtering criteria, and process this request input and information through an orchestration process, which includes, but is not limited to, ensuring that query input does not contain profane language, obtaining dialogue context from session storage, rewriting and generating queries, retrieving one or more results from a retrieval service, sending information to a generative machine learning model, and sending information to a response classifier to ensure that the response is free of bias, profane language, and defamation.

[0064] In some embodiments, the natural language generative application service 210 may implement session storage 214. Session storage 214 may be responsible for ensuring that the context of the conversation is maintained (e.g., even if the socket connection is closed by the user). Session storage 214 may also provide data on the conversation history (as described below). Figure 6 (As discussed above). Session storage 214 can also provide data for analysis (e.g., the number of queries per session, the number of active sessions at a given time, etc., as described above). Session storage 214 can use session IDs and message IDs to track each conversation and the associated threads of that conversation associated with each user ID (which may be specific to a particular end user of the generative application, which may have multiple different users).

[0065] In some embodiments, the natural language generative application service 210 may implement retrieval 215. Retrieval service 215 may support data retrieval from retrieval sources, as described below. Figure 10 This is discussed in detail. For example, retrieval service 215 may implement metadata storage (e.g., retrieval configuration data 1020 discussed below), which can be used to store all metadata associated with a particular retrieval. This metadata may be information related to access roles or other credentials, such as Identity and Access Management (IAM) roles, Virtual Private Network (VPN) information (for communicating with data sources in a VPN). Retrieval service 215 will retrieve data from the underlying retrieval source and associated data repository. In some embodiments, retrieval service 215 may have built-in integration with a data repository (e.g., a pre-built data retrieval tool), or it may support obtaining and applying information from the application creator to specify parameters / queries in order to build a data retrieval tool to retrieve data.

[0066] In various embodiments, database service 230 can be various types of data processing services that, in some embodiments, perform general or specialized data processing functions (e.g., analytics, big data queries, time-series data, graphical data, document data, relational data, structured data, or any other type of data processing operation) on data stored across multiple storage locations. For example, in at least some embodiments, database service 210 may include various types of database services (e.g., relational) for storing, querying, and updating data. Such services can be scalable and extensible enterprise-grade database systems. In some embodiments, queries may point to databases across database service 230 distributed across multiple physical resources, and the database system can be scaled up or down as needed. In different embodiments, the database system can work efficiently with various types and / or organizational database schemas. In some embodiments, clients / subscribers can submit queries or other requests (e.g., requests to add data) in various ways, such as interactively via an SQL interface to the database system or via an application programming interface (API). In other embodiments, external applications and programs can submit queries to the database system using Open Database Connectivity (ODBC) and / or Java Database Connectivity (JDBC) driver interfaces.

[0067] In some embodiments, database service 220 may be various types of data processing services for performing different functions (e.g., query or other processing engines for performing functions such as anomaly detection, machine learning, data lookup, or any other type of data processing operation). For example, in at least some embodiments, database service 230 may include a MapReduce service that creates a cluster of processing nodes that implement MapReduce functionality on data stored in one of the data storage services 240. Various other distributed processing architectures and technologies may be implemented through database service 230 (e.g., grid computing, sharding, distributed hashing, etc.). It should be noted that in some embodiments, data processing operations may be implemented as part of data storage service 230 (e.g., a query engine processing requests for specific data).

[0068] Data storage service 240 can implement different types of data storage for storing, accessing, and managing data on behalf of client 270 as a network-based service, enabling client 270 to operate the data storage system in a cloud or network computing environment. For example, in some embodiments, a data storage service 230 can be implemented as a centralized data storage, allowing other data storage services to access data stored in the centralized data storage for processing and / or storage within those other data storage services. Such data storage service 240 can be implemented as an object-based data storage and can provide storage and access to various object or file data storage for placing, updating, and obtaining data objects or files of various types, sizes, or series. Such data storage service 230 can be accessed via a programmatic interface (e.g., API) or a graphical user interface. Data storage service 240 can provide virtual block-based storage for maintaining data as part of data volumes that can be mounted or accessed similarly to local block-based storage devices (e.g., hard drives, solid-state drives, etc.) and can be accessed using block-based data storage protocols or interfaces (e.g., Internet Small Computer Interface (iSCSI)).

[0069] In various embodiments, data streaming and / or event services can provide resources to ingest, buffer, and process streaming data in real time, which may be a source of a data repository. In some embodiments, data streaming and / or event services may act as an event bus or other communication / notification for event-driven systems or services (e.g., events occurring on provider network 200 services and / or local systems or applications).

[0070] Generally, client 270 can encompass any type of client configurable to submit network-based requests to provider network 200 via network 280, including requests to materialized view management platform 210 (e.g., requests to create generative applications at a natural language generative application service). For example, client 270 may include a suitable version of a web browser, or may include plug-in modules or other types of code modules that can execute as an extension of or within an execution environment provided by the web browser. Alternatively, client 270 can encompass applications in provider network 200, such as generative applications (or their user interfaces), to implement various features, systems, or applications. (For example, using the Natural Language Generative Application Service 210 API to send natural language requests to perform various tasks (e.g., question answering, summarizing, or various other features discussed above). In some embodiments, such applications may include sufficient protocol support (e.g., for a suitable version of Hypertext Transfer Protocol (HTTP)) for generating and processing web-based service requests without necessarily implementing full browser support for all types of web-based data. That is, client 270 may be an application that can directly interact with provider network 200. In some embodiments, client 270 may generate web-based service requests based on a Representational State Transfer (REST) ​​style web-based service architecture, a document- or message-based web-based service architecture, or another suitable web-based service architecture.)

[0071] In some embodiments, client 270 may provide access to provider network 200 to other applications in a manner transparent to them. For example, client 270 may integrate with an operating system or file system to provide storage on one of the data storage services 240 (e.g., a block-based storage service). However, the operating system or file system may present different storage interfaces to applications, such as a conventional file system hierarchy of files, directories, and / or folders. In this embodiment, the storage system service model may be utilized without modifying the application. Instead, the details of interfacing with the data storage service 240 may be coordinated by client 270 and the operating system or file system on behalf of the application executing within the operating system environment.

[0072] Client 270 can transmit network-based service requests (e.g., natural language queries) to provider network 200 via network 280 and receive responses from provider network 200. In various embodiments, network 280 can encompass any suitable combination of networking hardware and protocols necessary to establish network-based communication between client 270 and provider network 200. For example, network 280 can typically encompass various telecommunications networks and service providers that jointly implement the Internet. Network 280 can also include private networks, such as local area networks (LANs) or wide area networks (WANs), and public or private wireless networks. For example, given client 270 and provider network 200, both may be provisioned within an enterprise with their own internal networks. In such embodiments, network 280 can include the hardware (e.g., modems, routers, switches, load balancers, proxy servers, etc.) and software (e.g., protocol stacks, accounting software, firewall / security software, etc.) necessary to establish network links between given client 270 and the Internet, and between the Internet and provider network 200. It should be noted that in some embodiments, client 270 may use a private network instead of the public Internet to communicate with provider network 200.

[0073] As described above, the natural language generative application service 210 can support communication with an external data source 290 via a network 280 in order to obtain data for performing various natural language tasks.

[0074] Figure 3This is a logical block diagram illustrating the interaction of creating a Natural Language Generation Application (NLP) at a Natural Language Generation Application Service, according to some embodiments. Application management 218 can support various requests for creating generational applications to perform natural language tasks using features of the Natural Language Generation Application Service 210. For example, application management 218 can support various features of the generational application to create web applications or other hosted applications. Even when the generational application itself is not hosted, unhosted applications can still be created to manage various backend features via requesting data, security, task orchestration, and other features for the generational application from frontend 211. Application management 218 can support the creation of generational applications, which can, for example, add any identity provider. The end user of the generational application should then be able to log in as the configured identity provider. In some embodiments, application management 218 can support the creation of custom headers on hosted generational applications (e.g., creating custom headers for web applications). Application management 218 can support adding custom prefixes to URLs or other web identifiers provided to access hosted generational applications. The created generative applications can support both managed and unmanaged applications, enabling chat / conversation interactions using the application's associated data repository and service-hosted generative machine learning models.

[0075] A request 302 to create an unmanaged application can be received. The creation request may include many of the aforementioned configuration features or parameters, such as identity providers, implementing or enabling various analytics collections, associating or specifying various associated data repositories, enabling / specifying various custom features (e.g., actions, styles, etc. as described above). Request processing 300 may be invoked by control plane 212 (which may be invoked by frontend 211, not shown) to execute the request and create configuration information for the unmanaged application 312 in application metadata 310. Various features of the unmanaged application can be changed in subsequent requests (not shown), such as adding or removing data repositories, adding, modifying, or removing custom features, or various other features of the unmanaged application. For the unmanaged application, application provisioning 320 may still assign an application identifier and / or other information, as indicated at 321. When the unmanaged generative language application 352 uses the provided identifier to invoke the Natural Language Generative Application Service 210 via frontend 211 to perform different tasks (e.g., in response to end-user interaction 354), as indicated at 356. Although not shown, an interaction with the identity provider may be performed prior to interaction 356 (e.g., direct interaction with the identity provider system / service via application 352). The end-user identity already determined by the identity provider (e.g., using login or other end-user identification procedures) may be included in information interaction 356 to be specific to the identified end-user.

[0076] In addition to adding managed application metadata 314, request processing 300 may also initiate application creation 305 for request 304 to create a hosted application. Application provisioning 320 may provision computing resources 330 and network endpoints for accessing the generative natural language application 332 (which can be configured according to various options supported by application management 218). For example, creation request 304 may include many of the aforementioned configuration features or parameters, such as identity providers, implementing or enabling various analytics collections, associating or specifying various associated data repositories, and enabling / specifying various custom features (e.g., actions, styles, etc. as described above). Various features of the unhosted application may be changed in subsequent requests (not shown), such as adding or removing data repositories, adding, modifying, or removing custom features, or various other features of the hosted application. Application provisioning 320 may obtain computing resources 330 (e.g., virtual computing resources used as host systems) from a computing service provider in provider network 200 and build the generative natural language application 332 according to the provided configuration features. For example, different software components corresponding to different selected features can be acquired and integrated based on application-specific information (e.g., identified data repositories, identified data retrievers, identity providers, etc.). The executable form of the generative application (e.g., compiled, assembled, or otherwise constructed) can then be installed as generative natural language application 332 on provisioned computing resources. A network endpoint (e.g., a network address, such as a URL) can be provided, allowing end users to access generative natural language application 332.

[0077] Once created, the generative natural language application 332 may be ready to accept end-user requests 344 and interact with the natural language generative application service 346 via frontend 211. An example interaction flow is described below. The end-user first accesses the network endpoint of the hosted generative application (e.g., a web application) and is redirected to a login page of a configured identity provider, where the end-user enters their username and password. After successful authentication, the end-user is directed to obtain access credentials for the generative natural language application 332 (e.g., using the SAMLRedirectAPI, where the identity provider provides a SAMLAssertion certificate, and then uses SAMLAssertion to call STS (Security Token Service) assumeRoleWithSAML to obtain sigv4 credentials (AccessKey, SecretKey)). The obtained credentials may be valid for a period of time (e.g., 1 hour) allowing the end-user access to the generative natural language application 332. The end-user is then directed to a homepage for final authentication and qualification (e.g., using cookies or other session-saved information). An authentication token can be obtained and used to establish connections for interactive features (e.g., network socket chat connections with frontend 211) and event streams by signing all calls with these credentials, and the connections and event streams can be stored in browser memory for further use until they expire.

[0078] Figure 4 This is a logical block diagram illustrating the interaction of adding a data repository according to some embodiments. A request 402 to add a repository and perform indexing can cause request processing 400 to initiate ingestion 410 to obtain 411 data from data source 401 and provide the ingested data 412 to index generation 420, which can generate an index based on a known pattern and store 409 the indexed data repository. Data repository metadata 430 can be updated 405 to add a new repository.

[0079] For example, ingestion 410 can implement different connectors (e.g., software components that interact with an agent or are deployed as agents) on data source 401. As mentioned above, data source 401 can be various types of data storage sources, processing sources, message sending and receiving sources, streaming sources, or other information sources, either inside or outside the provider network 200. Different connectors can implement different file interpreters, parsers, crawlers, or other features that can interpret and retrieve information from data source 401 for inclusion in the index. For example, ingestion 410 can extract metadata describing data objects (e.g., document-wide metadata describing authors, titles, publishers, etc.) and the data itself (e.g., as paragraphs of document text). Data extraction as part of ingestion 410 can be implemented as described below. Figure 8A and 8B The segmentation techniques are discussed in detail. For example, a document can be parsed and then segmented into paragraphs using a sliding window that starts at a certain position and includes all tags until the end of the window, without splitting or breaking sentences. However, in other embodiments, paragraph overlap or sentence segmentation within paragraphs can be implemented during extraction and indexing. Once acquired, the ingested data 412 can be provided to index generation 420 for index creation. Index generation 420 can implement various indexing techniques to perform data searching when performing natural language tasks, as described below. Figure 5 and 6 As discussed above, indexes can support natural language search and can use fields, vectors, or other representations to model the underlying extracted data to support data retrieval by the data retriever. Different types of indexes can be implemented in different embodiments. For example, sparse indexes can be created that index data for specific fields, including those data objects (e.g., documents) that have said fields.

[0080] A 404 request to add a repository but not index it can be performed by updating the data repository metadata (405) (and may include schema information for searching / accessing the data repository). For example, the request may provide location information, such as a network address, access credentials, data format, or other schema information, to allow the data retriever to obtain data for the retrieval pipeline when performing natural language tasks, as described below.

[0081] Figure 5This is a logical block diagram illustrating a data orchestration workflow for processing natural language requests, according to some embodiments. As described above, natural language task orchestration 213 can interact with different services of generative natural language service 210 to perform natural language tasks. For example, session storage 214 can be accessed to obtain dialogue history information for a given natural language request, data access management 217 can be accessed to obtain specific data retrieval user information to enforce access control to the associated data repository, and retrieval 215 can be invoked to retrieve relevant data. The following description provides examples of task orchestration workflows that can be executed by natural language task orchestration 213 for each received task, as requested by the generative application.

[0082] Natural language requests for natural language tasks can be received, as indicated at 504. The task orchestration workflow 500 can implement a dialogue history 510. The dialogue history 510 can retrieve (if any) past dialogues for decontextualization. For example, a query / search can be performed on the session store 220 using a user identifier and / or session identifier to find other requests performed for the end user of the generative application. Multiple past sessions can be retrieved (if any). In some embodiments, the number can be determined from a large number of stored dialogues, rounds, or tasks (e.g., n most recent dialogues) based on a window of past dialogues, rounds, or other tasks. Dialogue data can be retrieved and provided for further processing. If no dialogue history exists, entries, data structures, or files can be created to store the dialogue history (including current natural language requests and tasks 502).

[0083] Intent classification model 520 can be used to classify the intent of natural language requests, including tasks that are sent directly to prompt generation 540 and generative language model 550. In some embodiments, intent classification model 520 can be a rule-based model that selects different intent classifications based on heuristics or other rules indicating different intents (e.g., looking for mathematical operators or conjunctions in the request to determine whether it is a multipart request, such as "add the earnings summary of X to the cash flow report of Y to generate a consolidated financial summary" or "if strategy type X is available in state Y, then use the information of Z to generate strategy type X").

[0084] In some embodiments, the intent classification model 520 can be implemented using machine learning-based methods that can be implemented for the intent classifier model 122. For example, neural network-based language models, such as Bidirectional Encoder Representations from Transformer (BERT) or Robustly Optimized BERT Pre-training Method (ROBERTA). These or various other machine learning models can be trained to recognize different intents. For example, the intent classifier model 520 can be trained to recognize communicative intents (without requiring the data retrieval pipeline 530) to detect general conversational natural language requests such as “Hello” or “How are you?”. For instructional or command intents including requests such as “Write an email,” “Summarize text,” “Write an article,” etc., the intent classifier model 520 can be further trained to detect instructional intents (including general commands and conversational commands). The intent classification model 520 can also be trained to recognize keyword requests (which may be queries that only input keywords without other context). For example, keyword requests may lack sufficient semantics and may be very short or overly technical. These keyword requests may not use a generative model (e.g., data retrieval might suffice), or they may require some query rewriting to become semantically meaningful. For example, searching for the IP address "172.1.2.100" or searching for specific terms like "MX-52113" (which could be a product number). Multipart tasks can also be trained similarly.

[0085] For some tasks, a single (or multi-part) task can be handled through the retrieval enhancement generation pipeline 530. The intent classification model 520 can classify tasks into retrieval tasks and non-retrieval tasks, where retrieval tasks are processed through the retrieval pipeline 530, and non-retrieval tasks are sent directly to the prompt generation 540. In some embodiments, multi-part tasks may include multiple (e.g., 0 to...) nRetrieval tasks and non-retrieval tasks. Non-retrieval tasks can include general conversational interactions (e.g., "casual conversation," such as "Hello," "Welcome to ABC," etc.) and tasks that can be performed without data retrieval (e.g., "Please divide 50,000 by 5,000"). Retrieval tasks can include instructions (e.g., "summarize," "describe," etc.), keywords (e.g., common entities in a data repository), and questions (sometimes called "queries"). If a conversation history is available, it can be provided to a generative language model (e.g., LLM) to rewrite instructions, keywords, or questions based on the conversation history (e.g., replacing ambiguous terms that can be determined from the conversation history, such as replacing pronouns with names or entities, adding additional terms such as "X's product or Y's service," etc.). Rewriting prompts can cause the generative machine learning model to return a rewritten form of the natural language request to perform the task (e.g., instruction or question), incorporating ambiguities or other clarifications arising from the conversation history. In some embodiments, the query rewriter 532 can be skipped in the absence of a conversation history.

[0086] The application's main storage area 536 can be used to provide the local user credentials or information to be used when retrieving data at data retrieval 534 (as described below). Figure 11 The technique described allows mapping end-user service user identifiers of generative applications to local identifiers at various data repositories for ACL enforcement purposes. Data retrieval 534 can select (as indicated at 535) an appropriate data retriever (based on the application's configuration at creation or update time, as discussed above). Figure 3 (As discussed). Once the relevant data paragraphs have been obtained, they are provided to prompt generation 540.

[0087] In various embodiments, prompt generation 540 may implement a rule-based prompt generator that can generate prompts based on classification types (e.g., by completing corresponding prompt templates for each classification type), the prompts including a request and (if applicable) relevant data retrieved at pipeline 530 and a rewrite request at 532. A generative machine learning model 550 may be trained to generate a natural language response to the prompt generated at 540. In some embodiments, the generative machine learning model 550 may be an LLM, including a privately developed or maintained base model (FM) that can use millions or billions of parameters to generate a response to the prompt. As part of the prompt, a request may be included using the provided relevant data (retrieved via pipeline 530) such that the generative machine learning model 550 does not return a response generated by illusion. The generative machine learning model 550 may be hosted as part of a natural language generative application service or as a separate service of provider network 200. In some embodiments, generative application creation may support the selection of a specific generative machine learning model from multiple available models, including models hosted outside of provider network 200.

[0088] The results of the generative language model 550 can then be evaluated 560 to determine if they are complete (e.g., the last part of a multipart problem), or a validation check can be performed to determine if the results are valid (if invalid, an error or other fault indication can be sent)). For example, the natural language task orchestration 213 can track the number of parts of a task that have been completed and return to an earlier stage in the workflow 530 to perform additional stages (e.g., based on the output of the previous part, or not based on the previous output).

[0089] In some embodiments, the source of the retrieved data used to generate the results can be indicated (570). For example, as described above, the response can be annotated using comments or other indications from the retrieved documents (e.g., based on document-level metadata from which retrieved document paragraphs have been obtained). In some embodiments, an additional machine learning model trained to detect profane language or other inappropriate content can be invoked on the results to ensure that the results are not invalidated by inappropriate content. In some embodiments, a response 504 indicating that the question cannot be answered (e.g., due to inappropriate results or insufficient relevant data provided from the retrieval pipeline) can be sent. Otherwise, a response 504 can be sent based on a response generated by a generative machine learning model 550.

[0090] Figure 6This is a logical block diagram illustrating data retrieval using a segmented document index to enhance generative machine learning results, according to some embodiments. A natural language request 602 can be received. Retrieval 610 may apply different retrieval techniques, such as sparse retrieval techniques that generate vectors or other representations, and search 612 the data repository index 640. In some embodiments, a hybrid of sparse retrieval and density-based retrieval can be implemented. In some embodiments, a minimum (or specified) number of candidate paragraphs can be obtained after search 612.

[0091] Candidate paragraphs 612 can then be provided to dense re-grading 620, which can apply density-based techniques (e.g., encoding candidate paragraphs and comparing them with the encoded form of the natural language request 602 to determine relevance). Confidence scores of relevance (e.g., determined as part of the comparison process) can be used to grade the candidate paragraphs. For example, graded candidate paragraphs 630 can implement different categories or buckets corresponding to different confidence score ranges for low-relevance paragraphs 632, moderate-relevance paragraphs 634, and high-relevance paragraphs 636. Example buckets are merely illustrative, and grading can use different numbers, permutations, or terms (e.g., no buckets). In some embodiments, if the number of candidate results in buckets above low-relevance 632 does not meet a minimum requirement, an error message (e.g., indicating that the question cannot be answered) can be returned in response to the natural language request 602 instead of continuing to process the natural language request.

[0092] Although described and illustrated in the context of a network of providers implementing natural language generative application services. Figure 2-6 ,but Figure 2-6 The various components shown and described herein can be readily applied to other natural query language processing techniques, systems, or devices that assist in performing natural language queries on datasets. Therefore, Figure 2-6 This is not intended to limit other embodiments of a system that can implement natural language query processing. Figure 7 This is a high-level flowchart illustrating, according to some embodiments, various methods and techniques for implementing segmented document indexing that enhances generative machine learning results through data retrieval.

[0093] Various systems and devices can implement the methods and techniques described below, either individually or collaboratively. For example, as mentioned above... Figure 2-6 The described business intelligence services can be implemented using various methods. Alternatively, these methods can be implemented using combinations of different systems and devices. Therefore, the examples above and / or any other systems or devices described as performing the methods shown are not intended to limit other different components, modules, systems, or configurations of systems and devices.

[0094] As indicated at 710, in some embodiments, a natural language request to perform a natural language task may be received at the generative machine learning system. For example, a hosted or unhosted generative application may send a request to an interface of a generative machine learning service (e.g., via an API) to perform a natural language task. In some embodiments, the request may include an existing session (e.g., an existing or ongoing chat) or be identified by said existing session, which uses network communication features (e.g., tokens and / or cookies) and utilizes a bidirectional communication protocol. In some embodiments, the natural language task may not be received from the generative application, but rather directly via an interface, a programmatic interface (e.g., an API), a command line, or a graph.

[0095] As indicated at 720, in some embodiments, a search representation of a natural language request for performing a natural language task can be generated to retrieve data from one or more datasets comprising documents to perform the natural language task. Different retrieval techniques may influence the generation of the search representation. For example, sparse retrieval techniques may generate a representative vector, i.e., selecting distinct words from the natural language task request, or selecting “important” words to include in the sparse vector using a neural network of neural networks (e.g., an ML model approach). Similarly, for density-based techniques, the natural language request can be encoded into a representative or latent space to perform distance-based similarity determination. In some embodiments, a hybrid of sparse retrieval and density-based retrieval may be implemented to generate the search representation.

[0096] As indicated at 730, in some embodiments, a search can be performed on an index generated for one or more datasets to return multiple candidate document segments based on corresponding similarities to the search representation, wherein the index includes entries corresponding to different document segments, which are determined based on the number of tags used to segment individual documents in a plurality of documents into different document segments. For example, corresponding representations (e.g., vectors) can be maintained or generated for different document segments (e.g., paragraphs), and these representations are then compared with the search representation. A minimum number of candidate segments can be obtained (e.g., returning the top 100 most relevant paragraphs).

[0097] As indicated at 740, in some embodiments, candidate document portions can be graded based on a corresponding relevance analysis for a natural language request to perform a natural language task. For example, a secondary comparison, such as a density-based re-grader, can be implemented by comparing each candidate portion with the natural language request to perform the request, by encoding both the natural language request and the candidate portions and then determining their similarity based on their positions in the latent space.

[0098] As indicated at 750, in some embodiments, the ranking can include one or more candidate document portions as context for prompting a generative machine learning model trained to perform a natural language task. For example, the ranking can be selected based on the top... n A number of candidate parts are used as part of a generative machine learning model to provide hints. In other scenarios, error indications can be provided without invoking the generative machine learning model if a minimum number of candidate parts with the minimum confidence score are not obtained (e.g., indicating that a natural language request cannot be performed). Templates can be used to generate hints, with space reserved within the hint to include candidate parts selected based on the level. For example, as mentioned above... Figure 5 The rule-based prompt generator discussed can: map data to fields to be included in a prompt template for a task; and then include instructions to generate a response using the provided data.

[0099] As indicated at 760, in some embodiments, a response to a natural language request for performing a natural language task can be returned based on the results obtained through a prompting generative machine learning model. As described above, in some embodiments, other post-processing of the results can be performed, including source attribution, verification, appropriate response verification, and integrity checks of the task processing workflow.

[0100] As mentioned above Figure 1 and Figure 4 As discussed, as part of enhanced generative machine learning, data segmentation techniques can be implemented to appropriately resize data portions (e.g., documents) for efficient and relevant search and retrieval. Figure 8A This is a high-level flowchart illustrating various methods and techniques for generating indexes of segmented documents, based on some embodiments.

[0101] As indicated at 810, in some embodiments, a request may be received to add documents as a data repository for use in retrieval augmentation generation using generative machine learning models. (As stated above regarding...) Figure 4 As discussed, documents can be added using the requested indexing method, thereby allowing data connectors or other components to access the data as part of data ingestion and perform data indexing. Segmentation techniques can be performed as part of extracting data for data ingestion and indexing.

[0102] For example, as indicated at 820, in some embodiments, a document may be segmented into parts to be added to the index of a data repository. As indicated at 830, in some embodiments, a document may be parsed into tags. As indicated at 840, in some embodiments, tags are included in document portions without segmenting sentences, starting from the beginning of the document and using a sliding window with a specified threshold number of tags (e.g., 200 tags), until the threshold number of tags is reached. As indicated at 850, in some embodiments, the sliding window may move forward to the beginning of the next sentence in the document. For example, as... Figure 8B As shown in document 870, a sliding window 872 moves forward without splitting or segmenting sentences. However, in other embodiments, overlapping portions of paragraphs may be included in the index. In other embodiments, segmented sentences within a paragraph may be included in the index.

[0103] As indicated at 860, in some embodiments, portions of a document may be stored in an index along with metadata describing the document.

[0104] Since both tuning and retrieval-enhanced generation can be used to improve the performance of generative machine learning models, techniques that optimally combine tuning and retrieval-enhanced generation in various embodiments can provide even greater performance advantages. For example, as described below... Figure 9 In detail, if sharded templates are used for both the tuning technique and when the tuned generative machine learning model is invoked, the generation of hints for the machine learning model can improve the effectiveness of the tuning. The shared template can contain information acquired as part of data retrieval, allowing fine-tuning (and subsequent model invocations) to utilize the structured placement of the retrieved data (e.g., in context field 952 of shared template 950). Furthermore, by improving the performance of fine-tuning, the tuned generative machine learning model can provide better and more accurate results, thereby improving the performance quality of various computing systems, services, and applications that rely on a better-tuned generative machine learning model.

[0105] Figure 9A logical block diagram illustrating template-based tuning of a generative machine learning model for performing natural language tasks is shown according to some embodiments. Generative machine learning system 910 may be a standalone system or service implemented as part of a provider network (e.g., a natural language generative application service (e.g., service 210)) and / or any other system that utilizes generative machine learning techniques to invoke the tuned generative machine learning model 920. It may receive a natural language task 902 (e.g., a question, an instruction, or a combination of both). Natural language task 902 may be associated with a natural language generative application that can use an associated data repository to perform various natural language tasks for the user (as described below). Figure 3-6 (As discussed). In this way, natural language processing tasks can be performed in a relevant, safe, and verifiable manner using specific datasets of interest to the application.

[0106] Generative machine learning system 910 can use a tuned generative machine learning model 920. Although not shown, data can be retrieved to enhance the context for performing natural language task 902. The tuned generative machine learning model 920 can be a fine-tuned generative model. For example, as indicated at 930, model fine-tuning system 930 (which can be implemented together with or separately from generative machine learning system 910) can receive a pre-trained generative machine learning model 934 and a tuning dataset 932 and perform fine-tuning, as described below regarding... Figure 11 and 13 This is discussed in detail. As part of model fine-tuning, a shared template 950 for natural language tasks can be used. Different parts of the shared template 950 can be completed for different tasks. Such parts may include, but are not limited to, instructions 951, context 952, history 953, and queries 954, which may store parts of task 902, context (e.g., retrieved data), past dialogue history, and questions. Not all fields of the template can be completed for every prompt, as different tasks may have different characteristics. Once tuned, the model can be fed to a generative machine learning system 910, as indicated at 936. These techniques can allow the generation and delivery of multiple different tuned generative machine learning models. In some embodiments, a natural language task may be associated with a selected tuned generative machine learning model (among multiple tuned generative machine learning models).

[0107] A shared template 950 can also be used in the generative machine learning system 910 to identify and complete parts of the template to generate hints. For example, as mentioned above... Figure 5As discussed, intent categories can be determined to identify which parts of the shared template 950 should be completed (e.g., parts mapped to instruction, question, keyword query, or casual conversation categories). Then, template-based hints 904 can be used to obtain results 906 from the tuned generative machine learning model 920. The generative machine learning system 910 can return a response 908, which can be based on result 906 (and potentially other post-processing of results, such as source attribution and result validation).

[0108] Figure 10 This is a logical block diagram illustrating the interaction with a retrieval system for a generative natural language application service, according to some embodiments. The retrieval request handler 1010 (which can receive requests from...) Figure 2 The requests dispatched by the front end 211 can accept different retrieval requests. For example, a request to add a retriever 1001, a request to remove a retriever 1002, a request to update a retriever 1003, a request to describe a retriever 1004, or a request to list retrievers 1005 can trigger a corresponding read / write 1012 to the retriever configuration data 1020. As part of adding and updating retrievers in the retriever configuration data 1020, features such as the network location or endpoint of the data repository (e.g., indexed and unindexed repositories), authentication and authorization parameters, network configuration information (e.g., communicating with the data repository in a private network), access control information, retriever configuration (e.g., the number of items to retrieve), whether to paginate the results, data filtering, or various other data manipulations that can be performed on the retrieved data (e.g., reformatting the retrieved data).

[0109] Similar to the data connectors discussed above, different data retrievers 1030 can implement corresponding interface components to generate data retrieval requests and send them to the appropriate data repository. For example, one type of data retriever 1030 can read data from one type of index, and another type of data retriever 1030 can read data from another type of index (or external data repository).

[0110] A request 1006 invoking a selected retriever can cause a retrieval request handler 1010 to dispatch a query 1014 to a selected data retriever 1030, which has access to data in data repository 1040 1032. The schema information for some data repositories may be known (as it may have been generated during the ingestion of the indexed data repository). For other external or otherwise unindexed / uningested data repositories, schema information (which may be provided as part of an add retriever request 1001) can be used to search for and return relevant data. (As mentioned above...) Figure 5 As discussed, these data retrievers can be... Figure 2The natural language task orchestration 213 call.

[0111] Figure 11 This is a logical block diagram illustrating template-based tuning of a generative machine learning model according to some embodiments. As indicated at 1110, training data for different natural language tasks supported by a natural language generative application managed / created / supported by the natural language generative application service 210 can be obtained. Tuning dataset curation 1120 can identify and pull task mixes (e.g., based on various ratios or proportions of natural language tasks, which may vary depending on the determined workload / mixture of different natural language tasks of the generative machine learning model). In some embodiments, tuning dataset curation 1120 can augment the training data or add additional information to the training data, or generate additional training for different natural language tasks. For example, to achieve different ratios of natural language tasks, some additional training examples for a specific natural language task can be created as part of training dataset curation.

[0112] A tuning dataset 1130 with a specified task mix can be provided to model fine-tuning 1140. Model fine-tuning 1140 can use the tuning dataset 1130 to perform fine-tuning techniques to adjust the parameters of the generative machine learning model. A shared prompt template 1160 and a model 1150 to be tuned can also be provided. For example, when performing fine-tuning techniques, model fine-tuning can use one or more shared prompt templates to generate prompts for different natural language tasks. As described above, fine-tuning refers to further training performed on a given machine learning model, which can adapt the parameters of the machine learning model to a specific knowledge domain or task by using additional training data. In this case, model fine-tuning can be performed based on both the tuning dataset 1130 and the different shared prompt templates 1160 to formulate generated prompts, and then the results of the prompts are evaluated and used as part of model refinement to adjust the model parameters. When complete, the tuned generative machine learning model 1170 can be provided or deployed to a generative application performing a natural language task.

[0113] Although described and illustrated in the context of a network of providers implementing natural language generative application services. Figure 10 and 11 ,but Figure 10 and 11 The various components shown and described herein can be readily applied to other natural query language processing technologies, systems, or devices that provide generative language applications. Therefore, Figure 10 and 11 This is not intended to limit other embodiments of a system that can implement natural language query processing. Figure 12This is a high-level flowchart illustrating, according to some embodiments, various methods and techniques for template-based tuning of generative machine learning models used to perform natural language tasks.

[0114] Various systems and devices can implement the methods and techniques described below, either individually or collaboratively. For example, as mentioned above... Figure 2-8B The generative application services described in 10-11 can implement various methods. Alternatively, combinations of different systems and devices can implement these methods. Therefore, the examples above and / or any other systems or devices described as performing the methods shown are not intended to limit other different components, modules, systems, or configurations of systems and devices.

[0115] As indicated at 1210, in some embodiments, natural language requests to perform natural language tasks can be received via an interface of a generative machine learning system. For example, a hosted or unhosted generative application may send a request to an interface of a generative machine learning service (e.g., via an API) to perform a natural language task. In some embodiments, the request may include an existing session (e.g., an existing or ongoing chat) or be identified by said existing session, which uses network communication features (e.g., tokens and / or cookies) and utilizes a bidirectional communication protocol. In some embodiments, the natural language task may not be received from the generative application, but rather directly via an interface (e.g., a programmatic interface (e.g., an API), command line, or graphical user interface).

[0116] As indicated at 1220, in some embodiments, as part of generating prompts to perform a natural language task, the portion of a shared template to be completed can be identified. For example, as described above regarding Figure 6 The intent classification model discussed can be rule-based or a trained machine learning model capable of classifying intents for natural language tasks. Intent classification can be mapped to one of many different templates corresponding to different intent classifications. As mentioned above... Figure 7 As mentioned, multiple shared templates may be available; therefore, intent classification can be used to select which shared template and which fields(s) within the shared template. A rule-based suggestion generator can complete the suggestion using the identified portions of the shared template, along with other data, such as data retrieved from one or more associated data repositories for data retrieval tasks (and without data for non-retrieval tasks).

[0117] As indicated at 1230, in various embodiments, the generative machine learning model can be instructed to execute prompts generated based on a shared template. The generative machine learning model may have been tuned (as described above regarding...). Figure 9 and Figure 7 (As discussed) to perform multiple natural language tasks, including the natural language task described above. In some embodiments, the generative machine learning model may have been tuned by using a tuning dataset to complete the identified portion of a shared template for the natural language task.

[0118] As indicated at 1240, in some embodiments, a response to a natural language request can be returned via an interface of the generative machine learning system, the response being at least in part based on results received from the generative machine learning model. As described above, in some embodiments, other post-processing of results can be performed, including source attribution, verification, appropriate response verification, and integrity checks of the task processing workflow.

[0119] Figure 14 This is a high-level flowchart illustrating various methods and techniques for implementing template-based tuning of generative machine learning models, according to some embodiments. As indicated at 1310, in some embodiments, data mixtures used for training to perform different natural language tasks can be acquired to be included in the tuning dataset. As indicated at 1320, in some embodiments, the generative machine learning model can be fine-tuned using the tuning dataset to perform different natural language tasks. As indicated at 1330, in some embodiments, prompts can be generated based on different parts of a shared template that are identified for completion for different natural language tasks. For example, as described above regarding... Figure 7 As discussed, different training inputs from a tuning dataset can be obtained and used to generate cues. The results generated by the generative machine learning model can then be evaluated using the ground truth values ​​from the tuning dataset, and the results can be used to update the parameters of the generative machine learning model.

[0120] In various embodiments, the methods described herein can be implemented by any combination of hardware and software. For example, in one embodiment, the methods can be implemented by a computer system (e.g., the computer system in Figure 1410), the computer system including one or more processors that execute program instructions stored on a computer-readable storage medium coupled to a processor. The program instructions can be configured to implement the functions described herein (e.g., implement the functions of various servers and other components of a network-based virtual computing resource provider described herein). The various methods shown in the figures and described herein represent example embodiments of the methods. The order of any method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0121] The embodiments of segmented document indexing described herein, which enhance generative machine learning results through data retrieval, can be executed on one or more computer systems that can interact with a variety of other devices. The embodiments of template-based tuning of generative machine learning models for performing natural language tasks, described herein, can also be executed on one or more computer systems that can interact with a variety of other devices. Figure 14 One such computer system is illustrated. In various embodiments, computer system 1400 can be any of a variety of devices, including but not limited to personal computer systems, desktop computers, laptop computers, notebook or netbook computers, mainframe computers, handheld computers, workstations, network computers, cameras, set-top boxes, mobile devices, consumer devices, video game consoles, handheld video game devices, application servers, storage devices, peripheral devices such as switches, modems, routers, etc., or generally any type of computing device, computing node, computing node, computing system, computing system, or electronic device.

[0122] In the illustrated embodiment, computer system 1400 includes one or more processors 1410 coupled to system memory 1420 via input / output (I / O) interface 1430. Computer system 1400 also includes a network interface 1440 coupled to I / O interface 1430, and one or more input / output devices 1450, such as cursor control device 1460, keyboard 1470, and display 1480. Display 1480 may include a standard computer monitor and / or other display systems, technologies, or devices. In at least some embodiments, input / output device 1450 may also include a touch- or multi-touch-enabled device, such as a tablet or input pad, through which a user types input using a stylus-type device and / or one or more fingers. In some embodiments, embodiments may be implemented using a single instance of computer system 1400, while in other embodiments, multiple such systems or multiple nodes constituting computer system 1400 may host different portions or instances of the embodiment. For example, in one embodiment, some elements may be implemented via one or more nodes of computer system 1400 that are different from those implementing other elements.

[0123] In various embodiments, computer system 1400 may be a single-processor system including one processor 1410, or a multiprocessor system including several processors 1410 (e.g., two, four, eight, or another suitable number of processors). Processor 1410 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 1410 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs) such as x86, PowerPC, SPARC, or MIPS ISA or any other suitable ISA. In a multiprocessor system, each of processors 1410 may typically, but does not necessarily, implement the same ISA.

[0124] In some embodiments, at least one processor 1410 may be a graphics processing unit (GPU). A GPU can be considered a dedicated graphics rendering device for a personal computer, workstation, game console, or other computing or electronic device. Modern GPUs can be highly efficient in manipulating and displaying computer graphics, and their highly parallel architecture makes them more efficient than a typical CPU for a range of complex graphics algorithms. For example, a GPU may implement graphics primitive operations in a manner that makes performing multiple graphics primitive operations much faster than drawing directly to the screen using a host central processing unit (CPU). In various embodiments, graphics rendering may be implemented at least in part by program instructions configured to execute on one of such GPUs or in parallel on two or more such GPUs. The GPU may implement one or more application programming interfaces (APIs) that allow programmers to invoke the functions of the GPU. Suitable GPUs are commercially available from vendors such as NVIDIA Corporation and ATI Technologies (AMD).

[0125] System memory 1420 may store program instructions and / or data accessible to processor 1410. In various embodiments, system memory 1420 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In the illustrated embodiment, program instructions and data implementing the desired functionality described above are shown stored in system memory 1420 as program instructions 1425 and data storage device 1435, respectively. In other embodiments, program instructions and / or data may be received, transmitted, or stored on different types of computer-readable media or on similar media separate from system memory 1420 or computer system 1400. Generally, non-transitory computer-readable storage media may include storage media or memory media, such as magnetic or optical media, for example, a disk or CD / DVD-ROM coupled to computer system 1400 via I / O interface 1430. Program instructions and data stored via a computer-readable medium can be transmitted via a transmission medium or signal (e.g., an electrical signal, an electromagnetic signal, or a digital signal), which can be transmitted via a communication medium such as a network and / or a wireless link, for example, via a network interface 1440.

[0126] In one embodiment, I / O interface 1430 can coordinate I / O traffic between processor 1410, system memory 1420, and any peripheral devices within the device, including network interface 1440, or other peripheral interfaces such as input / output device 1450. In some embodiments, I / O interface 1430 can perform any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., system memory 1420) into a format suitable for use by another component (e.g., processor 1410). In some embodiments, I / O interface 1430 may include support for devices attached via various types of peripheral buses, such as the Peripheral Component Interconnect (PCI) bus standard or variants of the Universal Serial Bus (USB) standard. In some embodiments, the functionality of I / O interface 1430 may be divided into two or more separate components, such as a northbridge and a southbridge. Additionally, in some embodiments, some or all of the functionality of I / O interface 1430, such as the interface for system memory 1420, may be directly incorporated into processor 1410.

[0127] Network interface 1440 allows data to be exchanged between computer system 1400 and other devices (such as other computer systems) attached to the network, or between nodes of computer system 1400. In various embodiments, network interface 1440 may support: communication via wired or wireless general data networks (such as any suitable type of Ethernet); communication via telecommunications / telephone networks (such as analog voice networks or digital fiber optic communication networks); communication via storage area networks (such as fiber optic SANs); or communication via any other suitable type of network and / or protocol.

[0128] In some embodiments, the input / output device 1450 may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for typing or retrieving data by one or more computer systems 1400. Multiple input / output devices 1450 may be present in the computer system 1400 or distributed across different nodes of the computer system 1400. In some embodiments, similar input / output devices may be separate from the computer system 1400 and may interact with one or more nodes of the computer system 1400 via a wired or wireless connection (such as through network interface 1440).

[0129] As shown in FIG1410, memory 1420 may include program instructions 1425, which can implement various methods and techniques as described herein, and data storage device 1435, including various data accessible by program instructions 1425. In one embodiment, program instructions 1425 may include software elements as described herein and as illustrated in the accompanying drawings. Data storage device 1435 may include data that can be used in an embodiment. In other embodiments, other or different software elements and data may be included.

[0130] Those skilled in the art will understand that computer system 1400 is illustrative only and is not intended to limit the scope of the techniques described herein. Specifically, computer systems and apparatuses may include any combination of hardware or software capable of performing the indicated functions, including computers, personal computer systems, desktop computers, laptop computers, notebook or netbook computers, mainframe computers, handheld computers, workstations, network computers, cameras, set-top boxes, mobile devices, network devices, internet-connected appliances, PDAs, wireless mobile phones, pagers, consumer devices, video game consoles, handheld video game devices, application servers, storage devices, peripherals such as switches, modems, routers, etc., or generally any type of computing or electronic device. Computer system 1400 may also be connected to other devices not shown, or alternatively may operate as a standalone system. Furthermore, in some embodiments, the functionality provided by the illustrated components may be combined in fewer components or distributed across additional components. Similarly, in some embodiments, the functionality of some illustrated components may not be provided and / or other additional functions may be available.

[0131] Those skilled in the art will also understand that while various items are shown as being stored in memory or on a storage device during use, these items or portions thereof may be transferred between memory and other storage devices for memory management and data integrity purposes. Alternatively, in other embodiments, some or all of the software components may be executed in memory on another device and communicate with the illustrated computer system via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or portable article of manufacture for retrieval by a suitable driver, examples of which are described above. In some embodiments, instructions stored on a non-transitory computer-accessible medium separate from computer system 1400 may be transmitted to computer system 1400 via a transmission medium or signal (such as electrical, electromagnetic, or digital signals transmitted via communication media such as networks and / or wireless links). Various embodiments may also include receiving, transmitting, or storing instructions and / or data implemented according to the foregoing description on a computer-accessible medium. Therefore, the invention can be practiced with other computer system configurations.

[0132] It should be noted that any distributed system embodiment or any component thereof described herein can be implemented as one or more network services. For example, a leader node within a data warehouse system can present data storage services and / or database services to clients as network-based services. In some embodiments, the network-based service can be implemented by software and / or hardware systems designed to support interoperable machine-to-machine interactions on a network. The network-based service can have an interface described in a machine-processable format, such as a Network Services Description Language (WSDL). Other systems can interact with the network service in a manner specified by the description of the network-based service's interface. For example, the network-based service can define various operations that other systems can invoke, and can define specific application programming interfaces (APIs) that other systems can expect to follow when requesting various operations.

[0133] In various embodiments, a network-based service can be requested or invoked using a message that includes parameters and / or data associated with the network-based service request. Such messages can be formatted according to a specific markup language such as Extensible Markup Language (XML), and / or encapsulated using a protocol such as Simple Object Access Protocol (SOAP). To execute a network service request, a network-based service client can use an internet-based application-layer transport protocol such as Hypertext Transfer Protocol (HTTP) to assemble a message including the request and transmit the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the network service.

[0134] In some embodiments, web services may be implemented using expressive state transfer (“RESTful”) techniques instead of message-based techniques. For example, web services implemented according to RESTful techniques may be invoked via parameters included in HTTP methods such as PUT, GET, or DELETE, rather than being encapsulated in SOAP messages.

[0135] The accompanying drawings and various methods described herein represent example embodiments of the methods. The methods can be implemented in software, hardware, or a combination thereof. The order of the methods can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0136] Various modifications and alterations can be made, as will be apparent to those skilled in the art to which this disclosure pertains. The invention is intended to cover all such modifications and alterations; therefore, the above description is to be considered illustrative rather than restrictive.

[0137] Embodiments of this disclosure may be described in accordance with the following terms: Clause 1. A system comprising: Multiple computing devices, each implementing at least one processor and memory, implement a natural language generation application service, which is configured to: Receive natural language requests to perform natural language tasks; Generate a search representation of the natural language request to perform the natural language task, in order to obtain data from one or more datasets comprising multiple documents to perform the natural language task; Access the index generated for the one or more datasets and perform a search to return multiple candidate document segments based on corresponding similarity to the search representation, wherein the index includes multiple entries corresponding to different document segments, the different document segments being determined based on the number of tags used to segment each of the multiple documents into the different document segments; The ranking of the candidate document portions is generated based on the corresponding relevance analysis for the natural language request to perform the natural language task. Based on the stated level, one or more candidate document portions are selected as context to prompt the generative machine learning model trained to perform natural language tasks; and The response to the natural language request for performing the natural language task is returned based on the results obtained by prompting the generative machine learning model.

[0138] Clause 2. The system according to Clause 1, wherein the search representation and the search are performed according to sparse retrieval techniques, and wherein the grading of the candidate document portions according to the corresponding relevance analysis for the natural language request is performed according to density-based grading.

[0139] Clause 3. The system according to any one of Clauses 1 to 2, wherein the natural language generative application service is further configured to: Receive a request to add one or more datasets for data retrieval when performing a natural language request using the generative machine learning model; Segmenting the individual documents among the plurality of documents, wherein, for the purpose of segmenting the documents, the natural language generative application service is configured to: The multiple documents are parsed into tags; and Starting from the beginning of each document in the document and using a sliding window with a specified threshold number of tags, tags are included in the document portion without segmenting the documents into sentences until the threshold number of tags is reached; and The individual documents, which are segmented from the plurality of documents, are stored in the index as the different parts of the plurality of documents.

[0140] Clause 4. The system according to any one of Clauses 1 to 3, wherein the natural language generation application service is implemented as part of a provider network, and wherein the natural language request for performing the natural language task is received from a natural language generation application created and hosted at the natural language generation application service.

[0141] Clause 5. A method comprising: Receive natural language requests to perform natural language tasks at the generative machine learning system; The generative machine learning system generates a search representation of the natural language request to perform the natural language task, in order to obtain data from one or more datasets comprising multiple documents to perform the natural language task; The generative machine learning system performs a search on an index generated for the one or more datasets to return multiple candidate document segments based on corresponding similarity to the search representation, wherein the index includes multiple entries corresponding to different document segments, the different document segments being determined based on the number of tags used to segment each of the multiple documents into the different document segments; The generative machine learning system classifies the candidate document portions based on a corresponding relevance analysis to the natural language request for performing the natural language task. The generative machine learning system uses one or more candidate document portions, including those from the candidate document portions, as context to prompt the generative machine learning model trained to perform natural language tasks; and The generative machine learning system returns a response to the natural language request for performing the natural language task based on the results obtained by prompting the generative machine learning model.

[0142] Clause 6. The method according to Clause 5, wherein the search representation and the search are performed according to sparse retrieval techniques, and wherein the grading of the candidate document portions according to the corresponding relevance analysis for the natural language request is performed according to density-based grading.

[0143] Clause 7. The method according to any one of Clauses 5 to 6, wherein the different document portions do not overlap.

[0144] Clause 8. The method according to any one of Clauses 5 to 7 further includes: Receive a request to add one or more datasets for data retrieval when performing a natural language request using the generative machine learning model; Segmenting the documents among the plurality of documents, wherein segmenting the documents includes: Parse the multiple documents into tags; and Starting from the beginning of each document in the document and using a sliding window with a specified threshold number of tags, tags are included in the document portion without segmenting the documents into sentences until the threshold number of tags is reached; and The individual documents, which are segmented from the plurality of documents, are stored in the index as the different parts of the plurality of documents.

[0145] Clause 9. The method according to Clause 8, wherein storing the individual documents segmented from the plurality of documents includes storing document-level metadata obtained from the plurality of documents.

[0146] Clause 10. The method according to any one of Clauses 5 to 9, wherein classifying the candidate document portions according to a corresponding relevance analysis for the natural language request performing the natural language task includes distributing the individual candidate document portions into corresponding buckets associated with different relevance confidence levels.

[0147] Clause 11. The method according to Clause 10 further includes, before prompting the generative machine learning model, determining that the number of candidate document portions in the bucket with the lowest relevance confidence in the respective bucket does not meet the minimum number requirement.

[0148] Clause 12. The method according to any one of Clauses 5 to 11, wherein the generative machine learning system is a natural language generative application service, and wherein the natural language request for performing the natural language task is received from a natural language generative application created at the natural language generative application service.

[0149] Clause 13. The method of Clause 12, wherein the index is created in response to a request received at the natural language generation application service and is associated with the natural language generation application.

[0150] Clause 14. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive natural language requests to perform natural language tasks at the generative machine learning system; Generate a search representation of the natural language request to perform the natural language task, in order to obtain data from one or more datasets comprising multiple documents to perform the natural language task; Perform a search on the indexes generated for the one or more datasets to return multiple candidate document segments based on corresponding similarity to the search representation, wherein the indexes include multiple entries corresponding to different document segments, the different document segments being determined based on the number of tags used to segment each of the multiple documents into the different document segments; The ranking of the candidate document portions is generated based on the corresponding relevance analysis for the natural language request to perform the natural language task. Based on the stated level, one or more candidate document portions are selected as context to prompt a generative machine learning model trained to perform natural language tasks; and The response to the natural language request for performing the natural language task is returned based on the results obtained by prompting the generative machine learning model.

[0151] Clause 15. One or more non-transitory computer-readable storage media as described in Clause 14, wherein the search representation and the search are performed according to a hybrid sparse retrieval and density-based retrieval technique, and wherein the grading of the candidate document portions according to the corresponding relevance analysis for the natural language request is performed according to a density-based grading technique.

[0152] Clause 16. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 15, wherein the different document portions do not overlap.

[0153] Clause 17. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 16, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: Receive a request to add one or more datasets for data retrieval when performing a natural language request using the generative machine learning model; Segmenting the individual documents among the plurality of documents, wherein, in order to segment the documents, the program instructions cause the one or more computing devices to perform the following actions when segmenting the individual documents among the documents: Parse the multiple documents into tags; and Starting from the beginning of each document in the document and using a sliding window with a specified threshold number of tags, tags are included in the document portion without segmenting the documents into sentences until the threshold number of tags is reached; and The individual documents, which are segmented from the plurality of documents, are stored in the index as the different parts of the plurality of documents.

[0154] Clause 18. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 17, wherein storing the respective documents segmented from the plurality of documents includes storing document-level metadata obtained from the plurality of documents.

[0155] Clause 19. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 18, wherein, when generating the rank, the candidate document portion, based on a corresponding relevance analysis for a natural language request performing the natural language task, the program instructions cause the one or more computing devices to implement distributing each candidate document portion of the candidate document portion into corresponding buckets associated with different relevance confidence levels.

[0156] Clause 20. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 19, wherein the generative machine learning system is a natural language generative application service, and wherein the natural language request to perform the natural language task is received from a natural language generative application created at the natural language generative application service.

[0157] Clause 21. A system comprising: A plurality of computing devices, each including at least one processor and a memory, are configured to implement a generative machine learning system, the generative machine learning system being configured to: Receive natural language requests for performing natural language tasks via the interface; Identify one or more parts of the shared template to be completed based on the classification determined in response to the natural language request; Generate prompts to perform the natural language task, wherein one or more parts identified in the shared template have been completed; The generative machine learning model is instructed to execute the prompts generated based on the shared template, wherein the generative machine learning model is tuned to perform multiple natural language tasks, including the natural language task, and the tuning is based on: using a tuning dataset to complete the identified portions of the shared template for the multiple natural language tasks; The interface returns a response to the natural language request, the response being at least in part based on results received from the generative machine learning model.

[0158] Clause 22. The system according to Clause 21, wherein the generative machine learning system is further configured to search one or more data repositories when generating the prompt to obtain data to be included in the context portion of the shared template.

[0159] Clause 23. The system according to any one of Clauses 21 to 22, wherein the generative machine learning model is tuned based on a specified mixture of different natural tasks included in the tuning dataset.

[0160] Clause 24. The system according to any one of Clauses 21 to 23, wherein the generative machine learning system is a natural language generative application service provided by a provider network, and wherein the natural language request is received from a natural language generative application created at the natural language generative application service.

[0161] Clause 25. A method comprising: Receive natural language requests to perform natural language tasks via the interface of the generative machine learning system; As part of generating prompts to perform the natural language task, one or more parts of a shared template to be completed are identified; The generative machine learning model is instructed to execute the prompts generated based on the shared template, wherein the generative machine learning model is tuned to perform multiple natural language tasks, including the natural language task, and the tuning is based on: using a tuning dataset to complete the identified portions of the shared template for the multiple natural language tasks; The natural language request is returned via the interface of the generative machine learning system, the response being at least in part based on the results received from the generative machine learning model.

[0162] Clause 26. The method according to Clause 25 further includes searching one or more data repositories when generating the prompt to obtain data to be included in the context portion of the shared template.

[0163] Clause 27. The method according to any one of Clauses 25 to 26, wherein the prompt is generated without retrieving data from one or more data repositories to complete a portion of the prompt.

[0164] Clause 28. The method according to any one of Clauses 25 to 27, wherein identifying one or more portions of the shared template to be completed as part of generating a prompt to perform the natural language task includes determining an intent classification of the natural language request to perform the natural language task, wherein the intent classification is mapped to one or more portions of the shared template to be completed.

[0165] Clause 29. The method according to any one of Clauses 25 to 28, wherein at least a portion of the shared template is not completed as part of generating the prompt.

[0166] Clause 30. The method according to any one of Clauses 25 to 29, wherein the shared template includes an instruction section, a context section, a history section, and a query section.

[0167] Clause 31. The method according to any one of Clauses 25 to 30, wherein the generative machine learning model is one of a plurality of generative machine learning models available for performing a natural language task, and wherein the generative machine learning model is selected to perform the natural language request for the natural language task.

[0168] Clause 32. The method according to any one of Clauses 25 to 31, wherein the generative machine learning model is tuned based on a specified mixture of different natural tasks included in the tuning dataset.

[0169] Clause 33. The method according to any one of Clauses 25 to 32, wherein the generative machine learning system is a natural language generative application service provided by a provider network, and wherein the natural language request is received from a natural language generative application created at the natural language generative application service.

[0170] Clause 34. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive natural language requests for performing natural language tasks via the interface; As part of generating prompts to perform the natural language task, one or more parts of a shared template to be completed are identified; The generative machine learning model is made to execute the prompts generated based on the shared template, wherein the generative machine learning model is tuned to perform multiple natural language tasks, including the natural language task, and the tuning is based on: using a tuning dataset to complete the identified portions of the shared template for the multiple natural language tasks; The interface returns a response to the natural language request, the response being at least in part based on results received from the generative machine learning model.

[0171] Clause 35. One or more non-transitory computer-readable storage media as described in Clause 34, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further implement: searching one or more data repositories when the prompt is generated to obtain data to be included in the context portion of the shared template.

[0172] Clause 36. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 35, wherein the prompt is generated without retrieving data from one or more data repositories to complete a portion of the prompt.

[0173] Clause 37. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 36, wherein, when identifying one or more portions of the shared template to be completed as part of generating prompts to perform the natural language task, the program instructions cause the one or more computing devices to perform: determining an intent classification of the natural language request to perform the natural language task, wherein the intent classification maps to the one or more portions of the shared template to be completed.

[0174] Clause 38. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 37, wherein at least a portion of the shared template is not completed as part of generating the prompt.

[0175] Clause 39. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 38, wherein the generative machine learning model is tuned based on a specified mixture of different natural tasks included in the tuning dataset.

[0176] Clause 40. One or more non-transitory computer-readable storage media according to any one of Clauses 34 to 39, wherein the generative machine learning system is a natural language generative application service provided by a provider network, and wherein the natural language request is received from a natural language generative application created at the natural language generative application service.

Claims

1. A system comprising: Multiple computing devices, each implementing at least one processor and memory, implement a natural language generation application service, which is configured to: Receive natural language requests to perform natural language tasks at the generative machine learning system; Generate a search representation of the natural language request to perform the natural language task, in order to obtain data from one or more datasets comprising multiple documents to perform the natural language task; Perform a search on the indexes generated for the one or more datasets to return multiple candidate document segments based on corresponding similarity to the search representation, wherein the indexes include multiple entries corresponding to different document segments, the different document segments being determined based on the number of tags used to segment each of the multiple documents into the different document segments; The candidate document portions are graded based on the corresponding relevance analysis for the natural language request to perform the natural language task. Using one or more candidate document portions from the candidate document portions as context, the system prompts the generative machine learning model trained to perform natural language tasks; and The response to the natural language request for performing the natural language task is returned based on the results obtained by prompting the generative machine learning model.

2. The system of claim 1, wherein the search representation and the search are performed according to a sparse retrieval technique, and wherein the grading of the candidate document portions according to the corresponding relevance analysis for the natural language request is performed according to a density-based grading.

3. The system according to claim 1 or claim 2, wherein the natural language generative application service is further configured to: Receive a request to add one or more datasets for data retrieval when performing a natural language request using the generative machine learning model; Segmenting the individual documents among the plurality of documents, wherein, for the purpose of segmenting the documents, the natural language generative application service is configured to: The multiple documents are parsed into tags; and Starting from the beginning of each document in the document and using a sliding window with a specified threshold number of tags, tags are included in the document portion without segmenting the sentences of the document until the threshold number of tags is reached; and The individual documents, which are segmented from the plurality of documents, are stored in the index as the different parts of the plurality of documents.

4. The system according to any one of claims 1 to 3, wherein the natural language generation application service is implemented as part of a provider network, and wherein the natural language request for performing the natural language task is received from a natural language generation application created and hosted at the natural language generation application service.

5. A method comprising: Receive natural language requests to perform natural language tasks at the generative machine learning system; The generative machine learning system generates a search representation of the natural language request to perform the natural language task, in order to obtain data from one or more datasets comprising multiple documents to perform the natural language task; The generative machine learning system performs a search on an index generated for the one or more datasets to return multiple candidate document segments based on corresponding similarity to the search representation, wherein the index includes multiple entries corresponding to different document segments, the different document segments being determined based on the number of tags used to segment each of the multiple documents into the different document segments; The generative machine learning system classifies the candidate document portions based on a corresponding relevance analysis to the natural language request for performing the natural language task. The generative machine learning system uses one or more candidate document portions, including the candidate document portions, as context to prompt the generative machine learning model trained to perform natural language tasks. as well as The generative machine learning system returns a response to the natural language request for performing the natural language task based on the results obtained by prompting the generative machine learning model.

6. The method of claim 5, wherein the search representation and the search are performed according to a sparse retrieval technique, and wherein the grading of the candidate document portions according to the corresponding relevance analysis for the natural language request is performed according to a density-based grading.

7. The method of claim 5 or claim 6, wherein the different document portions do not overlap.

8. The method according to any one of claims 5 to 7, further comprising: Receive a request to add one or more datasets for data retrieval when performing a natural language request using the generative machine learning model; Segmenting each document among the plurality of documents, wherein segmenting the documents includes: Parse the multiple documents into tags; and Starting from the beginning of each document in the document and using a sliding window with a specified threshold number of tags, tags are included in the document portion without segmenting the documents into sentences until the threshold number of tags is reached; and The individual documents, which are segmented from the plurality of documents, are stored in the index as the different parts of the plurality of documents.

9. The method of claim 8, wherein storing the individual documents segmented from the plurality of documents includes storing document-level metadata obtained from the plurality of documents.

10. The method of any one of claims 5 to 9, wherein classifying the candidate document portions according to a corresponding relevance analysis for a natural language request performing the natural language task comprises distributing each candidate document portion into a corresponding bucket associated with a different relevance confidence level.

11. The method of claim 10, further comprising, before prompting the generative machine learning model, determining that the number of candidate document portions in the bucket with the lowest relevance confidence in the respective buckets has not reached a minimum requirement.

12. The method according to any one of claims 5 to 11, wherein the generative machine learning system is a natural language generative application service, and wherein the natural language request for performing the natural language task is received from a natural language generative application created at the natural language generative application service.

13. The method of claim 12, wherein the index is created in response to a request received at the natural language generation application service and is associated with the natural language generation application.

14. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive natural language requests to perform natural language tasks at the generative machine learning system; Generate a search representation of the natural language request to perform the natural language task, in order to obtain data from one or more datasets comprising multiple documents to perform the natural language task; Perform a search on the indexes generated for the one or more datasets to return multiple candidate document segments based on corresponding similarity to the search representation, wherein the indexes include multiple entries corresponding to different document segments, the different document segments being determined based on the number of tags used to segment each of the multiple documents into the different document segments; The ranking of the candidate document portions is generated based on the corresponding relevance analysis for the natural language request to perform the natural language task. Based on the level, one or more candidate document portions are selected as context to prompt the generative machine learning model trained to perform natural language tasks. as well as The response to the natural language request for performing the natural language task is returned based on the results obtained by prompting the generative machine learning model.

15. One or more non-transitory computer-readable storage media according to claim 14, wherein the search characterization and the search are performed according to a hybrid sparse retrieval and density-based retrieval technique, and wherein the grading of the candidate document portions according to the corresponding relevance analysis for the natural language request is performed according to a density-based grading.

Images