Contextual pruning for search extension generation

Abstract

Provides contextual pruning for generating search extensions. [Solution] The context pruning system searches for passages that have been initially selected and ranked by an initial passage selection model, such as a model that determines similarity based on cosine similarity between a query and various passages. The context pruning system reranks the passages using a reranking model that can be used to prune clauses from the passages. Alternatively, the context pruning system can prune clauses from the passages using a different clause selection model. The clauses saved during the pruning process can be used to prompt a natural language processor for context-based results, and the context-based results can trigger subsequent actions such as displaying an interface that uses the context-based results or operating a robot that physically moves or speaks using the context-based results.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence (AI), and more specifically, to a system and method for efficiently prompting a natural language processor to generate more accurate responses.

Background Art

[0002] Users and applications are integrated with a natural language processor, such as a large language model (LLM), to process natural language queries. The large language model encodes the query considering billions of parameters, generates a query vector embedding that captures the meaning of the tokens in the query, constructs an understanding of the information regarding the query using the tokens, and processes this information in deeper layers to perform complex operations and understand and enhance the meaning with other tokens in a multi-layer neural network. Although the large language model is a powerful tool, it is computationally expensive.

[0003] The large language model is restricted based on what information is publicly available, and sometimes there is too much publicly available irrelevant information and it is restricted. Therefore, the large language model can be prompted with additional information and instructions to adjust the large language model to be more acceptable and generate results based on better information. Determining the additional information and instructions to provide to the large language model for a given query can be computationally expensive and / or may require manual input or custom-coded human expertise based on the given use case.

[0004] High-capacity and low-latency processes may be incompatible with computationally expensive and / or human-intervening technologies, and such processes often fail to deliver meaningful results because the specific expertise required by the query is lacking in the large-scale language model. [Overview of the project] [Problems that the invention aims to solve]

[0005] In various embodiments, a context pruning system retrieves passages that have been initially selected and ranked by an initial passage selection model, such as a model that determines similarity based on cosine similarity between a query and various passages. The context pruning system re-ranks the passages using a re-ranking model that can be used to prune clauses from the passages. Alternatively, the context pruning system can prune clauses from the passages using a separate clause selection model. The clauses saved in the pruning process can be used to prompt a natural language processor with context-based results, and the context-based results can trigger subsequent actions, such as displaying an interface that uses the context-based results, or operating a robot that physically moves or speaks using the context-based results. [Means for solving the problem]

[0006] In one embodiment, a computer implementation includes the steps of accessing a query and determining one or more passages potentially relevant to the query from a content corpus containing multiple clauses of multiple passages, wherein each of the one or more passages contains a subset of three or more clauses. The computer implementation further includes the steps of encoding the subset of three or more clauses and determining a clause-to-query relevance score for each clause. The clause-to-query relevance score of at least one clause in the subset of three or more clauses may vary depending on the content of at least one other clause in the subset of three or more clauses. The computer implementation further includes the steps of removing one or more clauses from the subset of three or more clauses to generate a modified subset of clauses, at least in part on the clause-to-query relevance scores of one or more clauses in the subset of three or more clauses. The computer implementation further includes the steps of generating a prompt for a natural language processor. The prompt includes an instruction that is at least partially based on the query, and at least a portion of the modified section as a context for generating a response by the natural language processor. The computer implementation further includes the steps of prompting the natural language processor with the prompt, and accessing and storing the result generated by the natural language processor at least partially based on executing the prompt.

[0007] In various embodiments, the natural language processor includes a large-capacity language model.

[0008] In other embodiments, the step of encoding the subset of three or more sections includes, for each section of the subset of three or more sections: determining a token-to-query relevance score for each token in the section based at least in part on the token-to-query similarity between the token embedding of the token and the query embedding of the query; and determining a section-to-query relevance score for the section based at least in part on the token-to-query relevance score for each token in the section. The token embedding for at least one token in at least one section is based at least in part on the content of at least one section and at least one other section of the subset of three or more sections.

[0009] In another embodiment, the computer implementation method includes the steps of displaying an option to adjust a threshold for selecting a node to remove (i.e., prune), selecting the option to adjust the threshold, and generating an adjusted threshold. One or more nodes are selected for removal based on the adjusted threshold.

[0010] In the same or a different embodiment, the step of determining the one or more passages from the content corpus includes the step of determining the passage-query similarity between the vector embeddings of the plurality of passages and the vector embeddings of the queries.

[0011] In the same or different embodiments, a computer implementation method includes the steps of dividing a plurality of documents into a plurality of passages, each containing a sequence of sentences, and training a model to identify which of the plurality of passages is most relevant to each query, thereby generating a trained passage selection model. The step of determining the one or more passages uses the trained passage selection model.

[0012] In the same or a different embodiment, the computer implementation further includes the steps of generating a trained clause selection model by prompting a large language model with a number of prompts requesting the selection of the most relevant clauses, and by assigning weights to clause features based at least partially on the results of the prompts, thereby identifying which of a subset of clauses is most relevant to each query. The step of encoding the three or more subsets of clauses uses the trained clause selection model. In another embodiment, the trained clause selection model is trained to select the subset of clauses that is most relevant to each query in relation to the total amount of information represented in each subset of the clause subsets for a particular passage.

[0013] In the same or a different embodiment, the step of determining the one or more passages is at least in part based on one or more passage-query similarity scores determined using an initial passage selection model. The computer implementation further includes the steps of refining the one or more passage-query similarity scores using another passage selection model, and determining the one or more clause-query relevance scores for the one or more clauses using a clause selection model. The clause selection model is trained to prune clauses separately from the other passage selection model which is trained to determine the most relevant passages, and the other passage selection model integrates even more features than the initial passage selection model.

[0014] In the same or a different embodiment, the step of determining the one or more passages is at least in part based on one or more passage-query similarity scores. The computer implementation further includes the step of determining one or more section-query relevance scores for one or more sections and adjusting the one or more passage-query similarity scores using a combined machine learning model. The step of removing one or more sections from a subset of three or more sections to generate a modified subset of sections is at least in part based on one or more section-query relevance scores.

[0015] In the same or a different embodiment, the step of determining the one or more passages from the content corpus is obtained to obtain an initial rank for the two or more passages. The computer implementation further includes the step of simultaneously reranking the two or more passages and selecting the one or more clauses to remove. In a different embodiment, the computer implementation further includes the step of selecting the one or more clauses to remove based on a subset of clauses that have the highest relevance to the query compared to the total amount of information expressed in each subset of the clauses for a particular passage.

[0016] In one embodiment, a computer implementation method for optimizing clauses retrieved from a data store to determine an answer to a query includes the step of retrieving a first set of clauses from the data store, ranked by their relevance to the answer to the query. The computer implementation method further includes the step of processing the query and the first set of clauses, simultaneously pruning the first set of clauses to generate a subset of the first set of clauses, and generating a subset of clauses in a neural network that generates a reranking score for at least one passage containing the subset of the first set of clauses, based on its relevance to the answer to the query. The computer implementation method further includes the step of prompting a natural language processor to provide context for an answer to a query using the query and the subset of clauses. The neural network is trained to prune at least some clauses using a cross-encoder that encodes the query and candidate clauses in a training set and generates an output including a binary mask used to prune at least some of the candidate clauses for the query in the training set, and a reranking score generated for passages having the candidate clauses.

[0017] In various embodiments, the system includes one or more data processors that access one or more non-temporary computer-readable storage media, which store instructions causing one or more data processors to perform some or all of the methods disclosed herein when executed by one or more data processors.

[0018] In various embodiments, a computer program product stores instructions implemented in a non-transitory machine-readable storage medium configured so that one or more data processors perform some or all of the methods disclosed herein.

[0019] The described technology can be implemented as a method performed by a machine, a memory, one or more processors, a machine or system including one or more non - transient computer - readable media storing instructions that cause the one or more processors to perform the steps of the method when executed, and / or one or more non - transient computer - readable media storing processor - executable instructions that cause the one or more processors to perform the steps of the method when executed.

[0020] Further application areas of the present disclosure will become apparent from the detailed description, the claims, and the drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of the present disclosure.

Brief Description of the Drawings

[0021] Hereinafter, various embodiments will be described with reference to the drawings. Note that the drawings are not drawn to scale, and elements having the same structure or function are denoted by the same reference numerals throughout the drawings. Also note that the drawings are only for facilitating the description of the embodiments and do not completely describe the present disclosure content nor limit the scope of the present disclosure content. [Figure 1a] It is a flowchart of an exemplary process for efficiently and robustly pruning a context for retrieval augmented generation. [Figure 1b] It is a flowchart of an exemplary process for efficiently and robustly pruning a context for retrieval augmented generation. [Figure 2a] It is a system diagram showing a context pruning system for training a context pruning subsystem that efficiently and robustly prunes a context for retrieval augmented generation. [Figure 2b]A system diagram showing a context pruning system that efficiently and robustly prunes context for search expansion generation. [Figure 3] An exemplary diagram of a user interface showing an option to adjust a threshold used to efficiently and robustly prune context for search expansion generation. [Figure 4] A diagram showing a distributed system that efficiently and robustly prunes context for search expansion generation. [Figure 5] An exemplary diagram showing a computer system that efficiently and robustly prunes context for search expansion generation.

Mode for Carrying Out the Invention

[0022] An explanation of efficient and robust context pruning for search expansion generation is provided in the following section.

[0023] · Generation of content using large language models and other natural language processors · Expansion of natural language queries by context related to the query · Model training for re-ranking and / or pruning of context using clause-level analysis · Query and context-based pruning · Exemplification of context pruning · Pruning Multiple Passage · Tuning the level of Pruning · Improve efficiency and accuracy with extraction-based pruning without incurring additional round-trips of generative abstraction • Compact architecture for identifying and pruning potentially relevant contexts Context pruning across domains Concurrently ranking and pruning contexts. • Triggering Downstream Activity as a Result of Context-Informed Natural Language Processing Computer System Architecture The steps described in each section can be started or completed in the order that provides the information used when the steps are performed. The functions in each section can be started or completed in the order that provides the information used when the functions are performed. All steps or items of a function can be performed by a personal computer system, a cloud computer system, a local computer system, a remote computer system, a single computer system, a distributed computer system, or any other computer system that provides the processing, storage, and linking resources used to perform the step or function item.

[0024] • Content generation using large-scale language models and other natural language processors Natural language processors include multilayer neural networks and other machine learning implementations for using natural language, determining semantic meaning in natural language, and responding to natural language with semantic-based responses. Large-scale language models (LLMs) such as GPT-4, GPT-3.5, Llama, Gemini, Claude, and Falcon are various examples of natural language processors. They provide deep layers based on a rich understanding of natural language across multiple domains and can be prompted by natural language queries instructing large-scale language models to generate responses. Responses can include all information available to the model (generally public information accessible via the internet), including not only all information provided in the instruction, but also underlying queries such as problem specifications, user queries, and problem-solving constraints explicitly stated in the instruction.

[0025] Large-scale language models and other natural language processors can interact with users or applications. Users can submit commands or user queries to the LLM via a web, application interface, or other user interface to generate responses that leverage a deep understanding of natural language requests and the vast amount of information available to the LLM. Users can authenticate with the LLM in a user session, for example, using a username and password, and the LLM can be configured to remember or not remember previous queries submitted by the user in order to answer queries after the user has submitted one.

[0026] An LLM session, which retains memory of previous queries and responses, can emulate a natural user interaction without the need to repeat information already provided. LLM is often used in chatbot implementations to improve chatbot functionality and handle open-ended questions that are not included in the chatbot's predefined paths.

[0027] Applications may submit instructions to the Large-Scale Language Model (LLM) via an Application Programming Interface (API) of the LLM, which is exposed for the use of the application. The API can authenticate the application in current and future sessions with the LLM using an authentication token and / or other shared secrets shared with the authenticated user who previously managed the application. Applications can submit queries using the API, which can provide a more appropriate response from the LLM by being propagated from the user query, modified with additional problem-solving constraints determined by the application, and / or supplemented with other information determined by the application.

[0028] • Extending natural language queries with context related to the query Instructions to an LLM may include contextual information or "context" that informs the LLM of specialized information related to a problem specification that is unavailable to the LLM or cannot be adequately highlighted by the LLM. The LLM uses the available information, as well as the context, to generate a context-based response to the instruction. If the context-based response does not apply to the use case, the application or user may modify the instruction to include additional context and / or problem-solving constraints until the LLM response applies to the use case. Such interactions between the LLM caller (user or application) and the LLM apply to many different scenarios and can generate sophisticated responses from the LLM that take specialized instructions and specialized contextual information into account.

[0029] The large amount of available contextual information may be initially narrowed down to relevant contexts based on how similar the context is to the application or user query. For example, cosine similarity or other similarity measures can be sought between the vector embeddings of the query and the vector embeddings of the contextual passage. A passage may contain body text consisting of one or more paragraphs or text sections, each paragraph or section may contain one or more clauses. A passage may also contain a title, which may be stored at the beginning of the body text or separately from the body text. As a result of initial high-level filtering, passages that are potentially relevant to the query may be retained, while passages that are not potentially relevant to the query may be filtered out beforehand. The fact that a passage is relevant to the query does not determine which parts of the passage should be added as context to the query. The most relevant parts of a passage may be added to support a guided answer to a query that takes into account the information presented in the most relevant parts of the passage.

[0030] While it's possible to add context to a query, adding too much context to a natural language query can cause confusion in the natural language processing system, such as overemphasizing redundant or irrelevant context, or being unable to highlight relevant context due to being embedded in less relevant context. Adding too much context can confuse the natural language processing system about the general instruction requested for the query, and too much context can cause the system to provide overly complex or nuanced answers to simple questions. For these and other reasons, pruning the context you add allows the natural language processing system to focus on answering queries based on relevant information without confusing it with irrelevant or redundant information.

[0031] Search augmentation generation improves various aspects of large-scale language model (LLM) generation, but it introduces problems such as overhead due to long contexts and the propagation of irrelevant search information into the generated response. Context pruning addresses both of these aspects by removing irrelevant parts from the searched context before LLM generation. Context pruning using the techniques described herein is efficient or powerful enough to be useful without requiring multiple round trips to the LLM for each query, in a wide range of scenarios such as when the amount of relevant information contained in the context is variable or of varying lengths, or when evaluated across various domains. In one embodiment, a context pruning system prunes and reranks the searched relevant context to answer the question efficiently and powerfully. The context pruning system can dynamically sense the amount of pruning required for a given context and can be used without additional customization or specialization for various domains. The context pruning system can approach the context pruning task in a sequence labeling manner and integrate context pruning and context reranking capabilities through training on various data. Contextual pruning systems can perform contextual pruning in various domains and environments with little to no loss of accuracy compared to computationally intensive methods, and with little to no additional latency in Retrieval Augmented Generation (RAG) pipelines.

[0032] RAG has become a widely used paradigm for improving the factual accuracy, attribute, and adaptability of Large-Scale Language Models (LLMs). By extending a given user query with the relevant context retrieved, it is possible to prevent the generation of false information and provide references used to generate answers. Furthermore, using domain-specific data stores allows access to and inference of previously unknown knowledge without tweaking the LLM. Another important advantage of the RAG approach is its plug-and-play architecture. Practitioners can select the components (searcher, generator LLM, etc.) that are best suited to their specific use case to optimize performance.

[0033] At the same time, using RAG can lead to computational overhead due to the increased input length, and irrelevant information present in the retrieved context may be propagated into the generated response. These problems can be solved by developing a more efficient and robust LLM by modifying the architecture to handle longer contexts more efficiently, or by increasing the diversity of the tuning data to improve the handling of irrelevant context. However, LLM tuning may be resource-intensive or not applicable to proprietary (closed) LLMs. An alternative solution involves pruning the retrieved context by removing irrelevant context portions from the user query, thereby reducing the context length and thus improving the generation speed of the LLM. Such context pruning can be used plug-and-play with all generator LLMs in the RAG pipeline.

[0034] In one embodiment, a context pruning system can strongly support zero, one, two, or more potentially relevant clauses to an input query, depending on the query, rather than a fixed number or ratio of clauses. In fact, a context can contain a variety of relevant information parts, from an empty context to a fully relevant context, and the context pruning system can adaptively sense such changing contexts. A context pruning system can also be used efficiently by avoiding multiple calls to an LLM per query. Instead of using an LLM of billions of elements as its base model, the context pruning system can use a more compact model (e.g., a BERT-based model). Furthermore, the context pruning system can not design an abstract context compresser with sequential autoregressive generation of the final context, but instead rely on extracting context clauses from the context without generating new context clauses. A BERT-based model can be used as an extracting pruner, i.e., one with a lightweight predictive head for selecting relevant context parts. Contextual pruning systems are robust in that they do not require training on separate datasets and can be applied to various data domains based on the model, system architecture, and training data used to first train the contextual pruning system.

[0035] The level of pruning granularity (i.e., token-level versus clause-level (also called sentence-level) pruning) may vary depending on the pruner. In one embodiment, a contextual pruning system uses clause-level pruning according to the query, which prunes semantic units (clauses or sentences) that are deemed irrelevant to generating the answer. Another approach is token-level pruning, which removes low-level grammatical units such as articles and interjections. Such pruning may or may not be performed independently depending on the query. Both approaches are independent and may be combined for token-level and clause-level pruning.

[0036] In one embodiment, the context pruning system is an adaptive, efficient, and robust clause-level context pruner for query responses that, once trained, can be used without customization across various domains and configurations. The context pruning system can generate a binary mask predicted by the context pruning system using binary sequence labeling. The binary mask can store decisions for clauses (from zero to all) relevant to the query. The context pruning system can be trained from a lightweight DeBERTa model based on various data. In certain embodiments, the processes between clause context pruning and passage reranking overlap, so that clause context pruning, combined with passage reranking, becomes a single process using a single model, eliminating the additional latency of context pruning in the RAG pipeline.

[0037] The context pruning system has been tested across various QA domains and demonstrates far superior performance compared to other approaches, showing little to no loss of accuracy, and offering uncustomizable applicability that allows for context pruning at virtually no cost. The context pruning system can sense the number of related sentences at every position within a context and is robust to various context lengths.

[0038] In various examples, a Search Enhancement Generation (RAG) pipeline can include five steps: (0) a question or query from the user or application; (1) a data store, i.e., a collection of documents (text portions) to search; (2) an efficient but approximate searcher that enables quick retrieval of potentially relevant passages from a large data store (which may be a dual-encoder model where queries and passages are encoded independently), using, for example, cosine similarity between query embeddings and data items in the data store (e.g., passages or paragraphs); (3) an expensive cross-encoder reranker that reduces and sorts the set of passages retrieved from the text retrieved by the approximate searcher (cross-encoding means encoding passages along with the query); and (4) a generator LLM that takes the user's query and selected relevant passages as input and outputs the final response. The context pruning system may, in step (3), include a combination of a pruner and a reranker that outputs a binary mask indicating which clauses should be retained in the passage and that determines the reordering or reranking of the passage, or it may include an independent context pruner between steps (3) and (4) after or before a separate reranking has been performed.

[0039] Figure 2b is a system diagram showing a context pruning system 200B that efficiently and robustly prunes context for generating search extensions. In the example shown in Figure 2b, a query or user question 214B such as "How much did Elon Musk pay for Twitter?" is received. The query's embedding can be used by the searcher 230 to search the data store 202 for passages with embeddings similar to the query. The passage 232 found in the example may contain the text: "Musk's acquisition of the social media company was controversial. <...>On October 28, he finally made the acquisition. Tesla CEO Elon Musk signed an agreement to acquire Twitter for the initially offered price of $54.20 per share. The total acquisition price was approximately $44 billion. Within days, Musk fired the top management. <...>" The retrieved passage 232 can be input into the context pruning subsystem 238, which prunes and reranks the passage to generate a reranked and pruned passage 234 containing the following aided text: "Tesla CEO Elon Musk has signed an agreement to acquire Twitter for the initially offered price of $54.20 per share. The total acquisition price is approximately $44 billion." The saved text can be provided to the natural language processor 240 as context along with query 214B, which can generate a response 236 to the query, such as "Elon Musk acquired Twitter for $44 billion."

[0040] • Model training for context re-ranking and / or pruning using clause-level analysis Figure 2a is a system diagram showing a context pruning system for training the context pruning subsystem 238 shown in Figure 2b, which efficiently and robustly prunes context for search extension generation. In the example shown in Figure 2a, the context content data store 202 can be used for search and reranking. In one example, search and reranking are performed using a pre-trained model. Search and reranking can be performed using a sample example training question 214A, such as "How many French words are there in English?". The example training question 214A is used to search passages using a basic model in the searcher and reranker 204 that finds similar content by comparing the content vector embeddings of the data store 202 with the vector embeddings of question 214A. In the early stages of search and reranking, the similarity threshold is low, so passages that may be similar to the query may be excessively included. During training, the context of the retrieved passage 208 can be provided as context to the Data Labeler Large Language Model (LLM) 206, for example, in example prompts 228 where different clauses are filtered by markers or input text such as "[]" or numbered using other markers or symbols that are less likely to appear in the clauses.

[0041] Question: How many French words are there in English? Context: [1] The influence of French on English. [2] The most notable influence of French is… Answer the question using only the information provided in the context. If no useful information is provided, output "No answer". If you use part of the context in your answer, you must quote the entire sentence using brackets []. Do not quote sentences not used in your answer. The data labeler LLM206 can generate training responses to training prompts using considerable computing time and resources, but such training responses can be generated without human intervention. For example, a powerful, comprehensive, larger, and more computationally intensive data labeler LLM such as Llama-3-8B can determine a ground truth label, and a smaller model with fewer parameters can be trained based on that label. Furthermore, the time and resources spent in the training step do not need to be spent using the larger language model in subsequent steps.

[0042] In one embodiment of the context pruning subsystem 238 shown in Figure 2b, the context pruning subsystem 216 of the context pruning system 200A is trained using Llama-3-8B to provide a reference truth label. Given a question 214A and a retrieved passage (context) 208, the passage can be divided into clauses, and Llama-3-8B can be prompted to select clauses relevant to the given question. This approach does not require manual input and allows for more efficient training than manual labeling. An example of a prompt 228 is shown in Figure 2a. Another example of a prompt instruction for Llama-3-8B is "Output the index of sentences relevant to the given question." Using such results as reference truth labels, the context pruning system leverages the LLM's powerful question-answering capabilities, instructing the LLM to answer the given question and quote relevant contextual sentences in its answer. In particular, the context pruning system 200A can instruct the LLM to answer a given question using only the information provided in a given context, and can output "No answer" if relevant information is not provided. The context pruning system 200A can also specify a parsable quotation format [i] and a numeric sentence containing the same marker in the context. The quoted sentence can be parsed using a regular expression.

[0043] Llama-3-8B will almost always respond based only on the given context, and in about 90% of cases, it can output a citation. If no citation is generated and "No Answer" is not present in the LLM output, it can be filtered out, but this is because the context contains relevant information, but LLM "forgot" to cite that information or failed to cite it correctly as instructed.

[0044] For training purposes, the LLM can be prompted in various ways to determine which contexts to prun and generate criterion truth labels. In one example shown in Figure 2a, the LLM is asked to quote a sentence from any context that answers an arbitrary question. In another example, the LLM may be asked to quote a sentence by listing all the information in the context relevant to the question. In yet another embodiment, the LLM may be asked to output an index of sentences that answer an arbitrary question. Various training can be performed using various scenarios and datasets.

[0045] As shown in Figure 2a, the reranker 204 generates a reranking score 210 and retrieved context 208, and the data labeler LLM 206 generates pruned or preserved context 212 to provide the context pruning subsystem with a baseline truth label for training. The context pruning subsystem 216 emulates the baseline truth label of the data labeler LLM and trains its model to approach the reranking score 210 of the searcher and reranker 204. Using the baseline truth label, the context pruning subsystem generates a token-specific score 226 for a given section by increasing or decreasing the weight of tokens similar to the query, depending on whether the data labeler LLM preserves or prunes the section. The total score 224 for the passage (e.g., passage-query relevance score) can be computed using a linear layer with the [BOS] token output embedded by the reranking head (described in more detail below). The total score 224 can be learned to approach the criterion truth label of the reranker 204. The token-specific score 226 forms a binary mask 222 for the question and clause 220. The binary mask 222 and the total score 224 can be combined as a model output, which can be used to determine which clauses and / or passages to prune or keep. During inference, as shown in Figure 2b, the context pruning subsystem 238 is trained like the context pruning subsystem 216 in Figure 2a, including a combination of pruner and reranker that generates a binary mask 222 indicating which clauses in the retrieved passage 232 should be kept, and a total score 224 to determine the reranking of the passage to output as a reranked and pruned passage 234.

[0046] As an example, context pruning can be performed in a sequence labeling task by fine-tuning a compact model, such as the DeBERTa model, to encode query-context pairs and output a binary mask used to filter out irrelevant contextual portions. Silver labels or baseline truth labels for training can be generated by LLama-3-8B (AI@Meta, 2024). In various examples, this model can handle various noises within the context and select an appropriate pruning ratio. Queries are encoded together with the contextual statements (cross-encoding), which can provide a richer representation compared to, for example, an extractive RECOMP model where queries and contextual statements can be encoded independently. Context pruning systems use lightweight encoders that are even more efficient than LLM-based or abstract schemes.

[0047] In one embodiment, the context pruning system in Figure 2a trains a model for context reranking and pruning using a set of training queries and a search data store. In a particular embodiment, the context pruning system uses a training set of the MS MARCO document ranking collection containing 370k queries. For further details on the MS MARCO collection, see Nguyen, T., Rosenberg, M., Song, X., Gao, J. Verified by Tiwary, S., Majumder, R., & Deng, L. (November 2016). MS MARCO: Human-Generated Machine Reading Comprehension Dataset. Accessed December 7, 2024, Microsoft Research. Retrieved from https: / / www.microsoft.com / en-us / research / publication / ms-marco-human-generated-machine-reading-comprehension-dataset / , the entire contents of which are incorporated for reference for any purpose as fully expressly stated herein. The MS MARCO collection or any other training data used may be a domain-diversified data store crawled from the web. This can be used to improve the robustness of the final model to various domains and to train searchers and rerankers. In one embodiment, a training set of natural questions (NQ, 87k queries) is provided by Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., Toutanova, K., Jones, L., Chang, M.-W., Dai, It can be used as a question in the form of AM, Uszkoreit, J., Le, Q., & Petrov, S. (2019). Natural Questions: Benchmarks for Question-Answering Studies. Retrieved from https: / / huggingface.co / datasets / facebook / kilt_tasks, the entire contents are incorporated herein by reference for any purpose as fully expressly stated herein.

[0048] The search data store can be formed by splitting MS MARCO documents or other various datasets into passages consisting of k consecutive sentences, where k is randomly selected within a range such as 1 to 10. The random selection of k improves the pruner's robustness despite the diversity of the searched context lengths. Each passage can be saved during the pruning process by adding a page title before it, as long as at least one section of the passage is preserved. For each query, the context pruning system searches the top 5 or top N relevant passages using a robust search pipeline that may include, for example, a SPLADE-v3 searcher and a DeBERTa-v3 reranker. Additional information on configuring the search pipeline can be found in Rau's documentation. D., Dejean, H., Chirkova, N., Formal, T., Wang, S., Nikoulina, V., & Clinchant, This can be verified by S. (2024). Bergen: A benchmark library for generating search extensions. Retrieved from https: / / arxiv.org / abs / 2407.01102, the entire contents of which are incorporated hereby for reference for any purpose as fully explicitly stated herein. Additional information on the SPLADE-v31 searcher can be found at Lassance. This can be verified by C., Dejean, H., Formal, T., & Clinchant, S. (2024). SPLADE-v3: This can be verified in the new baseline for SPLADE, and its entire content is incorporated herein by reference for all purposes as fully expressed herein. Additional information on the DeBERTa-v3 reranker can be found in Lassance, This can be seen in C., Dejean, H., Formal, T., & Clinchant, S. (2024). SPLADE-v3: A new baseline for SPLADE, its entirety is incorporated hereby for reference purposes only, as fully expressed herein. The final set of retrieved passages may naturally vary in relevance or irrelevance to the question due to the incompleteness of the search and the use of the top 5 or top N passages.

[0049] Context-based pruning systems can be trained using Py-Torch (Paszke et al., 2019) and the HuggingFace transformer (Wolf et al., 2020). For additional details on training with Py-Torch, see Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., Devito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., & Chintala, Provided in S.(2019). PyTorch: An imperative-style high-performance deep learning library, provided by Curran Associates Inc., the entire contents of this document are incorporated for reference for any purpose as fully expressly stated herein. Additional details on training with the HuggingFace transformer are provided by Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von. Platen, P., Ma, C., Provided in Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M.,… Rush, A. (2020). Transducers: State-of-the-art natural language processing. Q. Liu and D. Schlangen (Eds), Proceedings of the 2020 Conference on Empirical Methods of Natural Language Processing: System Demonstrations (pp. 38-45), Computational Linguistics Society. https: / / doi.org / 10.18653 / v1 / 2020.emnlp-demos.6, retrieved from https: / / aclanthology.org / 2020.emnlp-demos.6 dated December 7, 2024, and the entire contents of this document are incorporated as reference for any purpose as fully expressly stated herein.

[0050] In one embodiment, a compact model such as DeBERTav3 or other BERT-based models or compact models are used as pre-trained models for training an independent context-pruning system. For further details on the DeBERTav3 model, see He, P. Provided to Gao, J., & Chen, W. (2021). DeBERTaV3: An Improvement to DeBERTa with ELECTRA-style Pre-training via Gradient-Distributed Embedding Sharing, the entire contents of this document are incorporated for reference for any purpose as fully expressly stated herein.

[0051] In one embodiment, the context pruning system is combined with a reranking function. In this embodiment, training can be started with a cross-encoder that has already been trained based on DeBERTa-v3. For additional information on pre-trained cross-encoders, see Lassance, C. Provided in & Clinchant, S. (2023). Naver Labs Europe (SPLADE) @ TREC Deep Learning 2022. Retrieved from https: / / arxiv.org / abs / 2302.12574 on December 7, 2024, and its entire content is incorporated herein by reference for any purpose as fully expressly stated. The ranking head (classifier) ​​may be initialized with a pre-trained version, and a separate pruning head may be trained from scratch.

[0052] In an exemplary embodiment, the learning rate is 8 × 10⁻⁶ -6 The batch size can be set to 48, and the model can be trained in one epoch. In the case of co-training, there is a contradiction between pruning and re-ranking. In one example, the re-ranking normalization coefficient λ can be set to 0.05 or 0.05 ± 0.04, which is selected as the minimum value that does not significantly degrade re-ranking performance in the MS MARCO development set.

[0053] Contextual pruning systems can be evaluated on a variety of QA datasets. For example, such QA datasets may include, but are not limited to, the following:

[0054] • For natural single-hop questions, Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., Toutanova, K., Jones, L., Chang, M.-W., Dai, AM Uszkoreit, J., Le, Q., & Petrov, S. (2019). Natural Questions: A Benchmark for Question-Answering Studies. Retrieved from https: / / huggingface.co / datasets / facebook / kilt_tasks, the entire contents are incorporated herein by reference for any purpose as fully expressly stated herein.

[0055] • For TyDi QA single-hop questions, Clark, JH, Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020). TyDi QA: A typological benchmark for information retrieval query responses in various languages. Transactions of the Japan Society for Computational Linguistics, 8, 454-470. https: / / doi.org / 10.1162 / tacl_a_00317. Retrieved from https: / / aclanthology.org / 2020.tacl-1.30 on December 7, 2024, and its entire content is incorporated herein by reference for any purpose as fully expressly stated.

[0056] • For PopQA single-hop questions, Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023). When language models should not be trusted: An investigation into the effectiveness of parameterized and non-parameterized memory. A. Rogers, J. Boyd-Graber, & N. Okazaki (Eds.), Proceedings of the 61st Annual Meeting of the Japan Society for Computational Linguistics (Volume 1: Long Papers) (pp. 9802-9822). Japan Society for Computational Linguistics. https: / / doi.org / 10.18653 / v1 / 2023.acl-long.546. Retrieved from https: / / aclanthology.org / 2023.acl-long.546 on December 7, 2024, its entire content is incorporated herein by reference for any purpose as fully expressed herein.

[0057] • For HotpotQA multi-hop questions, Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, CD (2018). HotpotQA: A dataset for diverse and explainable multi-hop question answering. E. Riloff, D. Chiang, J. Hockenmaier, & J. Tsujii (Eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 2369-2380). The Japan Society for Computational Linguistics. https: / / doi.org / 10.18653 / v1 / D18-1259. Retrieved from https: / / aclanthology.org / D18-1259 on December 7, 2024, its entire content is incorporated herein by reference for any purpose as fully explicitly stated.

[0058] Datasets with datastores in various domains can also be considered, and include, but are not limited to, the following:

[0059] • For biomedical questions using PubMed as a data store, BioASQ, Nentidis, A., Katsimpras, G., Krithara, A., Lima-Lopez, S., Farre-Maduell, E., Gasco Sanchez, L., Krallinger, M., & Paliouras, G. (2023). BioASQ 2023 Abstract: The 11th BioASQ Challenge on Large-Scale Biomedical Semantic Indexes and Query Responses. BioASQ 2023 Challenge Proceedings (pp. 227-250). Springer. https: / / doi.org / 10.1007 / 978-3-031-42448-9_19. ISBN 978-3-031-42447-2, the entire contents thereof, are incorporated by reference for any purpose as fully expressly stated herein.

[0060] For questions regarding educational course logistics, which uses course syllabus as a data store, please refer to SyllabusQA. Fernandez, N., Scarlatos, A., & Lan, A. (2024). SyllabusQA: Course Planning Query Response Dataset. L.-W. Ku, A. Martins, & V. Srikumar (Eds), Proceedings of the 62nd Annual Meeting of the Japan Society for Computational Linguistics (Volume 1: Long Papers) (pp. 10344-10369). Japan Society for Computational Linguistics. https: / / doi.org / 10.18653 / v1 / 2024.acl-long.557. Retrieved from https: / / aclanthology.org / 2024.acl-long.557 on December 7, 2024, its entire content is incorporated herein by reference for any purpose as fully expressed herein.

[0061] • For news-related questions using Google Search News as context, RGB, Chen, J., Lin, H., Han, X., & Sun, L. (2024). Benchmarking large-scale language models in search extension generation. Proceedings of the AAAI Artificial Intelligence Conference, 38(16), 17754-17762. https: / / doi.org / 10.1609 / aaai.v38i16.29728. Retrieved on December 7, 2024, from https: / / ojs.aaai.org / index.php / AAAI / article / view / 29728, the entire content of which is incorporated herein by reference for any purpose as fully expressly stated.

[0062] In one embodiment, the context pruning system is Craswell, The system is trained using the document-level MS MARCO dataset provided by N., Mitra, B., Yilmaz, E., Campos, D., & Lin, J. (2021). MS MARCO: A benchmark for ranking models in large-scale data systems. Proceedings of the 44th International ACM SIGIR Information Retrieval Research and Development Conference (SIGIR '21), 1566-1576. Computing Machinery Society. https: / / doi.org / 10.1145 / 3404835.3462804. The entire contents of this document are incorporated herein by reference for any purpose as fully expressly stated herein. The context pruning system can be trained on contexts of varying lengths by dividing each page into superimposed chunks of N sentences or clauses, where N is a random number between 1 and 10 (with a higher probability of increasing as the context length increases). A page title can be added before each chunk to preserve the chunk's background. The resulting data store size can be a 34M passage. In one embodiment, the context pruning system uses a Wikipedia data store prepared in a similar manner. As an example, the Wikipedia data store uses the 2024 Wikipedia data dump, Lin, The data dump is processed using a script provided by J., Ma, X., Lin, S. -C., Yang, J. -H., Pradeep, R., & Nogueira, R. (2021). Pyserini: A Python toolkit for reproducible information retrieval research using rare and high-density representations. Proceedings of the 44th ACM SIGIR International Conference on Information Retrieval Research and Development (SIGIR '21), 2356-2362. Computer Machine Society. https: / / doi.org / 10.1145 / 3404835.3463238, the entire content of which is incorporated by reference for any purpose as fully expressly stated herein. Exemplary Wikipedia data is accessible from Castorini(nd). Wikicorpus Experiment. GitHub. The entire contents of https: / / github.com / castorini / pyserini / blob / master / docs / experiments-wiki-corpora.md are retrieved and incorporated for reference for any purpose as fully explicitly stated herein. A version of the Wikipedia datastore can be created with a passage of N clauses or sentences where N / 2 clauses or sentences overlap, in order to test the robustness of the context pruning system for various context lengths.

[0063] In the case of life science data, in one embodiment, the context pruning system is Hsia, We use the PubMed dataset version provided by J., Shaikh, A., Wang, Z., & Neubig, G. (2024). RAGGED: On the Informative Design of Search Extension Generators. arXiv preprint arXiv:2403.09040. https: / / arxiv.org / abs / 2403.09040, the entire content of which is incorporated herein by reference for any purpose as fully expressly stated herein, Hsia, Accessible from J.(nd). RAGGED dataset. Hugging Face. Searchable at https: / / huggingface.co / datasets / jenhsia / ragged. The RAGGED dataset contains 58 million passages extracted from PubMed Green. Each passage (chunk) is preceded by a page title.

[0064] In the case of lecture plan data, in one embodiment, the context pruning system uses SyllabusQA to divide each lecture plan (provided by the author) into 100-word passages.

[0065] The context retrieved by the searcher using the trained model can include all context potentially relevant to the answer to the query. Such context can be provided to a context pruning system, which can then determine which clauses to save from the prompt in the response to the query and which to prune in the large-scale language model.

[0066] Pruning can be performed at the clause level, such as the sentence level, or at the token level, such as words, subwords, or terms with different meanings. A clause is an independent phrase that may contain one, two, or more tokens linked together to form a meaning. Clauses can be separated from each other using periods, commas, semicolons, newlines, or other delimiters. Clauses can be detected based on delimiters or based on the occurrence of subjects and verbs given in the text structure depending on the linguistic composition. Using existing libraries or known techniques, text can be divided into delimited contiguous text chunks that are stored or pruned independently of clauses, sentences, or other adjacent delimited text chunks.

[0067] A token is a term or set of letters and symbols that make up a language and have meaning, like a word. Some words represent different possible meanings from one another. In this case, the occurrence of a word can be probabilistically associated with one or more tokens. Some tokens can change the understood meaning of other tokens by increasing or decreasing the probability that a word refers to one tokenized meaning or something else. For example, in the clause "I hit the ball with the bat," the "hit" and "ball" tokens can determine the specific probabilistic meaning of the term "bat" to be "bat_sports:baseball:equipment," compared to the probabilistic meaning of the term "bat" being "bat_animal:mammal:flying" in a sentence that does not use the "hit" or "ball" tokens.

[0068] This example involves inferring the meaning of a token based on other tokens in the same section, but the meaning of a token can also be inferred from the surrounding text in the same passage or the same paragraph of the passage, so that general terms are determined to enrich the meaning of a section by referring to more specific topics before and after, even if the specific topic is not referenced in other sections within the meaningful section. A passage refers to a collection of articles, books, chapters, documents, pages, or other texts from a specific source relating to a particular topic. A paragraph of a passage is a collection of sections that are textually separated as a paragraph and / or divided into sizes that occupy the space allocated to the paragraph (e.g., space occupied by a limited number of characters or words). In one embodiment, the sections of a paragraph of a passage are encoded along with knowledge of the meaning of other tokens in the passage or paragraph so that similar terms used in the passage or paragraph have similar meanings, and interchangeable terms (e.g., more general or specific variations) are semantically aligned.

[0069] Processing context at the token level uses the probabilistic meaning of terms inferred from the surrounding text. Tokens are treated as the most probable variation, and the tokenized meaning can be inferred or abstracted from the text expressed in natural language. Processing context at the sentence or clause level preserves the terminology and structure of a clause without transforming or reinterpreting the clause into a set of the most probable tokens. When processing context at the clause level, the terminology of a clause can be used to infer how closely the sentence relates to other content. It does not distinguish between the most and least relevant subtopics of a clause, nor does it abstract and represent sub-meanings of clauses that do not appear literally in the text, including related or unrelated clauses. For example, a passage with two clauses can be written as follows: "Tesla and Rivian manufacture electric cars in the United States. A range of 300 miles or more is common." Using token-level processing, the two clauses can be reduced to the following facts: "Tesla produces cars in America," "Rivian produces cars in America," and "The range of their electric vehicles is over 300 miles," or "Tesla and Rivian cars generally have a range of over 300 miles." The meanings of these statements can be maintained, aligned, or pruned, respectively.

[0070] Using clause-level processing, the two clauses include the following: Section 1, "Tesla and Rivian manufacture electric vehicles in the United States," is related to "electric vehicles," "Tesla - electric vehicle manufacturer," "Rivian - electric vehicle manufacturer," and "United States - country," based on the tokens appearing in the section and considering the surrounding text.

[0071] Section 2, “Driving distances of 300 miles or more are common,” is related to “electric vehicles,” “Tesla - electric vehicle manufacturer,” “Rivian - electric vehicle manufacturer,” “America - country,” and “fuel efficiency,” based on the tokens appearing in the section and considering the surrounding text.

[0072] Using clause-level processing, such clauses can be individually maintained, sorted, or pruned depending on their relevance to the incoming query. If the first query asks, "What is the range of most electric vehicles?", the clause processing system can prune the first clause, maintain the second clause, and optionally maintain the title or other first sentence of passages related to the first query. If the second query asks, "Who manufactures electric vehicles in America?", the clause processing system can prune the second clause, maintain the first clause, and optionally maintain the title or other first sentence of passages related to the second query.

[0073] In a similar query applying token-level processing, the token processing system can prune the first two facts ("Tesla manufactures cars in the United States" and "Rivian manufactures cars in the United States") while retaining the third fact, "Electric vehicles have a range of 300 miles or more." In the second query, the token processing system can prune the third fact while retaining the first two. In some scenarios, splitting facts into chunks as the token processing system understands them can result in loss of meaning, misrepresentation of facts, or presenting false information as true. Using more compression in token-level processing (e.g., a higher percentage of token removal) means that facts are represented with an even higher level of granularity, and as subtle meanings are removed, overgeneralizations or misrepresentations of facts may become even more general. For example, the phrase "a range of 300 miles or more is common" can be compressed into "Electric vehicles have a range of 300 miles," which has a slightly different meaning. Section-level processing, regardless of the compression level, does not re-represent or reconstruct the contents of a section midway through, thus reducing the risk of presenting false information as true.

[0074] A context pruning system can prune at the clause level by scoring each token in a clause as to whether it is relevant to the query. The context pruning system is trained by a DataLabeler large language model, and if the clause is identified as relevant by the DataLabeler large language model, it can assign higher weights to tokens that exist or are similar between the query and the context clause, and if the clause is identified as irrelevant by the DataLabeler large language model, it can assign lower weights to tokens that do not exist or are not similar between the query and the context clause. If a sentence has more tokens similar to the query than tokens not similar to the query, or if the number or percentage of similar tokens is above a threshold, the context pruning system can save the sentence. If a sentence has even more tokens not similar to the query than tokens similar to the query, or if the number or percentage of similar tokens is below (or less than or equal to) a threshold, the sentence can be pruned by the context pruning system.

[0075] A context pruning system can take a concatenation of a question and a retrieved context as input and output a token-specific binary label indicating whether each token (e.g., a token defined by a pre-trained model tokenizer) should be included in the selected context. In one embodiment, a special token can be inserted into the beginning of each sentence, and a sentence-specific label for the special token can be predicted based on the other token labels in the sentence. The context pruning system can be trained with a binary token-specific classifier that uses criterion truth labels obtained from silver data labeling.

[0076] In various embodiments, when a clause is saved for a given passage, a selection of the first (title) clause may be included to provide an introduction to the saved clause and to avoid ambiguity in the generator's understanding of the context.

[0077] In one embodiment, each token in a clause is scored with a binary value of 0 or 1, depending on whether the token should be preserved (1) or pruned (0). For example, tokens with high similarity to the query (or tokens with a similarity score above a threshold) may be displayed as preserved (1), while tokens with low similarity to the query (or tokens with a similarity score below a threshold) may be displayed as pruned (0). The score for each token can determine the most likely meaning of the clause by considering the surrounding text and query, referring to how the token is used in the clause, how it is examined before and after within the clause, or in other clauses in a paragraph or passage of text, or in relation to the query. This meaning based on the surrounding text and query can be used to generate a vector embedding of the token, and the vector embedding of the token can be used to determine the similarity score of the token. For example, similarity can be estimated using a linear layer (e.g., a layer applied on top of the token embedding) or other similarity measures that can be used to determine the similarity between the token and the query. The similarity score of a token is for the token itself, although the surrounding text is considered.

[0078] In one embodiment, since the token embedding is already calculated with the query in mind, a simple linear layer applied on top of the token embedding is used to generate token-specific scalar values. Such scalar values ​​can represent "token-specific relevance scores" rather than simple "token-specific similarity scores," as they can also take into account relevance to the query.

[0079] A clause can be evaluated for similarity to a query based on an overall similarity score determined from the token similarity scores for the tokens within the clause. A stored representation indicating whether a token is similar to a query can be stored as a binary mask for the clause containing the tokens; therefore, a clause with 10 tokens can have an array, a vector, a set of 10 binary numbers, or a merged binary number representing a set of binary numbers. If a clause has more ones than zeros, or if the number or percentage of ones is above a threshold, the clause is saved. If a clause has more zeros than ones, or if the number or percentage of ones is below a threshold (or if the number or percentage of zeros is above a threshold), the clause is pruned.

[0080] The total similarity score at the node level can be saved as 0 or 1 to indicate whether a node should be removed or saved. Alternatively, the total similarity score at the node level can represent the percentage of 0s or 1s in the tokens present in the node (e.g., 0.9), indicating that 90% of the tokens in the node are saved. The total similarity score at the node level can also be determined from the sum or mean of the token-level similarity scores; for example, if the token-level scores are 0.8, 0.6, 0.7, 0.8, and 0.6, the total similarity score at the node level would be 0.7. Regardless of whether it is determined as a percentage of 1 or as an mean, the non-binary total similarity score at the node level can be used to rank nodes.

[0081] The percentage of 1s or 0s for tokens in a section, or the average or total similarity score, can be stored as the total similarity score at the section level. In the example shown in Figure 2a, special tokens such as "BOS" (Beginning of Sequence) can be prefixed to a section or added in other ways to determine the total similarity score at the section level. The ranking score of a passage can be determined based on the total similarity score at the section level of the passage. Passages containing sections with a high similarity token ratio or a high total similarity score are ranked higher than passages containing sections with a low similarity token ratio or a low total similarity score. Saved sections of a saved passage can be saved in a list in rank order based on the passage's ranking score and / or the total similarity score of the sections.

[0082] Saved clauses can be returned to be included with the query in the natural language processing system's prompt, while pruned clauses can be excluded from the list of clauses returned to be included in the prompt. The prompt, containing the list of clauses and the query, can then be used by the natural language processing system to generate context-specific responses.

[0083] Some extractive context pruners can independently encode the query sentences and then select the top sentence with the embedding closest to the query embedding. In such embodiments, the independent processing of the query limits context understanding, and the context pruner is fixed to a number of clauses to retain (e.g., specified as a hyperparameter). In other embodiments, the context pruning system determines the number of clauses to retain (from zero to all) based on the total similarity score per clause, and the clauses can be encoded dependently to each other. That is, the semantic meaning of tokens in one clause can inform the semantic meaning of tokens in other clauses.

[0084] Some pruning systems rely on inefficient autoregressive generation of the final context, sometimes presenting false information that does not exist in the input context as true. In other embodiments, context pruning systems extract the context that exists in the input context to prevent such false information provision and inefficiency.

[0085] Returning to the examples in Figures 2a and 2b, the context pruning system can assign a score to each token in each retrieved clause, such as "1" if it is "similar to the query" and "0" if it is "not similar to the query". The scores for the tokens in a clause can be aggregated to determine whether the total score of that clause is above or below the threshold for clause preservation. If the total score exceeds the threshold, the clause is preserved; if the total score is below the threshold, the clause can be pruned. The total score is tracked for each preserved clause, and passages containing preserved clauses can be ranked and included in the context. For example, passages containing preserved clauses with high total scores can be included in the context earlier, and passages containing preserved clauses with low total scores can be included in the context later.

[0086] The saved context clause can be included in the natural language processing system's prompt along with the query, enabling the generation of a response that provides a context-based answer to the query. The pruned clause does not need to be included in the prompt to prevent malfunctions of the natural language processing system or the introduction of "red herring," where the system is led to process irrelevant information simply because the information was provided as context with the query.

[0087] In various embodiments, context pruning systems can be trained to perform token-level labeling (rounding clauses or sentences during inference) or sentence-level or clause-level labeling. In the former case, clause representations may be richer, but the model is trained to produce similar predictions even for tokens within a single clause. In the latter case, the content of a clause may be represented by a single embedding, which can limit the expressiveness of the representation. Token-level strategies can be used in some scenarios using some datasets, while clause-level strategies can be used in other scenarios using other datasets.

[0088] As an example, a context-pruning system benchmark can be run using the BERGEN benchmark library for RAG, with the recommended hyperparameter settings. For further details on the BERGEN benchmark, see Rau. This can be found in D., Dejean, H., Chirkova, N., Formal, T., Wang, S., Nikoulina, V., & Clinchant, S. (2024). Bergen: A benchmark library for generating search extensions. It is retrieved from https: / / arxiv.org / abs / 2407.01102, and its entire contents are incorporated as reference for any purpose as fully expressed herein. For each query, the top 5 or top N relevant passages can be retrieved using a powerful and robust search pipeline such as the SPLADE-v3 searcher and the DeBERTa-v3 reranker (except for RGB where Google search passages are provided by the data creator). For more information on SPLADE-v3, see Lassance, This can be found in C., Dejean, H., Formal, T., & Clinchant, S. (2024). SPLADE-v3: A new baseline for SPLADE, its entire content is incorporated by reference for all purposes as fully expressly stated herein. Queries added as prefixes are transmitted to LLama-2-7B-chat or strong LLM along with the relevant documents (full length or pruned length) to generate responses. Each assessment dataset may contain short keyword responses that can be used to assess responses using LLM-based assessments. For further details on LLM-based assessments, see Rau, D., Dejean, H., Chirkova, N., Formal, T., Wang, S., Nikoulina, V., & Clinchant, This can be seen in S.(2024). Bergen: A benchmark library for generating search extensions. Retrieved from https: / / arxiv.org / abs / 2407.01102 on December 7, 2024, its entire contents are incorporated as reference for any purpose as fully expressly stated herein. Compression may be part of the pruned context.

[0089] • Query and context-based pruning Figures 1a and 1b are flowcharts illustrating an exemplary process for efficiently and robustly pruning context for search extension generation. Process 100A in Figure 1a begins with block 104A, where the context pruning system accesses the query. In block 106A, the searcher of the context pruning system determines one or more passages from a content corpus containing multiple clauses from multiple passages that are potentially relevant to the query. One or more passages contain a subset of clauses. In block 108A, the context pruning subsystem of the context pruning system encodes the subset to determine a clause-query relevance score for each clause. The clause-query relevance score for at least one clause in the subset depends on the content of at least one other clause in the subset. In block 110, the context pruning subsystem determines if there are any clauses to remove (i.e., prune). If there are no clauses to remove, in block 112A, all remaining clauses are preserved. If there are clauses to remove, in block 116A, the context pruning subsystem removes one or more clauses from the subset based at least partially on one or more clause-query relevance scores for one or more clauses in the subset, thereby generating a modified subset. In block 120A, the context pruning system generates a prompt for the natural language processor. This prompt includes instructions based on the query and at least some of the saved clauses or modified subset, as context for response generation. Then, in block 122A, the natural language processor, such as a large language model, is prompted by this prompt. In block 124A, the context pruning system accesses and stores the results generated based at least partially on the natural language processor executing the prompt.

[0090] Process 100B in Figure 1b begins in block 102B, where the context pruning system, as shown in Figure 2a, trains a context pruning subsystem to prune at least some of the clauses using a cross-encoder that encodes the queries and candidate clauses of the training set, and generates an output that includes the binary mask used to prune at least some of the candidate clauses for the queries of the training set, and the reranking score generated for the passages having the candidate clauses. In block 104B, the context pruning subsystem of the context pruning system searches a first set of clauses ranked in the datastore based on their relevance to the answers to the queries. In various other embodiments, another search subsystem may perform the search for the clauses to be processed by the context pruning subsystem. In block 114B, the trained context pruning subsystem processes the queries and the first set of clauses to generate a subset of clauses. In one embodiment, the trained context pruning subsystem is a neural network. Block 114B includes performing blocks 116B and 118B simultaneously. In block 116B, the context pruning subsystem prunes the first set of clauses to generate a subset of the first set of clauses. In block 118B, the context pruning subsystem reranks at least one passage containing the set of clauses using its relevance to the answer to the query. Then, in block 122B, the context pruning system prompts a natural language processor with the query and the subset of clauses to answer the query by leveraging the context added from the subset of clauses.

[0091] In various embodiments, pruning may or may not include information about the received query regarding which contexts should be added. If the query is known to the context pruning system, a more specialized context specific to the query is determined, so that contexts unrelated to the query can be removed even if they do not overlap with other contexts. If the query is not known to the context pruning system, contexts are not pruned based on whether they are related to the query. In this embodiment, contexts may be pruned as either overlapping with other content or not providing new information, but it is not possible to determine how closely the information provided is related to the query. In other words, if an additional tokenized representation of a fact does not change the existing meaning of a tokenized representation of another fact, the additional tokenized fact representation can be omitted due to duplication, regardless of how many clauses the fact is in and how the fact is expressed in those clauses. If the additional tokenized representation of a fact does not overlap, the additional tokenized representation of the fact can be included as non-duplicate.

[0092] Pruning to avoid duplication or to represent all tokenized facts may omit important information not fully captured in the tokenized representation of the facts, which can be retained when pruning at the clause level based on the received query. When pruning at the clause level based on the received query, information unrelated to the query can be pruned. Clauses related to the query can be preserved, and unrelated clauses cannot. The similarity between each individual clause and the query can be determined based on cosine similarity or other similarity determination mechanisms between content embeddings.

[0093] The context pruning system can use the concatenation of a question and the retrieved passage to output the probability that each token is present in the final context, and a reranking score for the passage (or, in the case of an independent model, the probability that there is no reranking score). A threshold T can be specified to binarize the token probabilities (whether to maintain or not), where probabilities above the threshold are maintained, and probabilities below the threshold are pruned.

[0094] Examples of contextual pruning In various examples, a query is received from user or application input requesting a response that is at least partially generated by a natural language processing model or other large-scale language model. The query is combined with context and processed using the natural language processing model to produce a result. For example, one query might be, "What do you put in the bottom of a Shepherd's Pi?" A passage with the original context that is sufficiently similar to the query can be found. In this example, the full text could include the following:

[0095] Shepherd's pie. History. In early cookbooks, this dish was a way to use up leftovers of all kinds of roast meat, with mashed potatoes lining the sides and bottom of the pie dish and topped with a mashed potato crust. Variations and similar dishes. Other potato-topped pies include: The modern "Cumberland pie" is a version using beef or lamb and topped with breadcrumbs and cheese. In medieval and modern Cumbria, fruit and spiced meat were filled inside a pastry crust. In Quebec, a variation of cottage pie is called "pâté chinois," which is made with ground beef in the bottom layer, canned corn in the middle, and topped with mashed potatoes. There are also meatless vegetarian versions of "shepherd's pie," and vegan versions that do not use meat or dairy products. In the Netherlands, a very similar dish called "philosopher's stew" may include ingredients such as beans, apples, prunes, and applesauce. In Brazil, there is a dish whose name comes from the fact that cassava puree covers a layer of dried meat. A context pruning system can analyze this passage, discovered using a general similarity filter, and determine a subset (if any) of the clauses in that passage that are relevant to the query. In this example, the context pruning system would prune this context to the following text: "Shepherd's pie. In early cookbooks, this dish was a way to use up leftover roast meats of all kinds, with mashed potatoes lining the sides and bottom of the pie dish and topped with a mashed potato crust." In this example, the context pruning system would choose one sentence about shepherd's pie and remove sentences about other similar dishes rather than (a) removing all context, (b) removing too much context, (c) retaining too much context, (d) selecting the wrong context, or (e) presenting false information as true. The first clause, such as the title, is kept to provide background information for understanding the clause being saved.

[0096] As another example, a query is received: "Where does the sweetness of fruit come from?" We can find a passage with the original context that is sufficiently similar to the query. In the example, the complete text of the body is as follows:

[0097] "Sweetness. Many plant species produce glycosides that exhibit sweetness at much lower concentrations than sugar. The best-known example is glycyrrhizin, the sweetening component of licorice root, which is about 30 times sweeter than sucrose. Another commercially important example is stevioside, extracted from the South American shrub Stevia rebaudiana, which is about 250 times sweeter than sucrose. Other powerful natural sweeteners include sweet proteins such as thaumatin, found in the fruit of Catemfe, native to West Africa. Egg lysozyme, an antibiotic protein found in chicken eggs, also exhibits sweetness."

[0098] A contextual pruning system can leverage this passage, discovered using a general similarity filter, to determine a subset (if any) of clauses in the passage that are relevant to the query. For example, after the pruning process, the contextual pruning system might save all of the passage's text without pruning any clauses at all.

[0099] "Sweetness. Many plant species produce glycosides that exhibit sweetness at concentrations far lower than sugar. The best-known example is glycyrrhizin, a sweetening compound found in licorice root, which is about 30 times sweeter than sucrose. Another commercially important example is stevioside, extracted from the South American shrub *Stevia rebaudiana*, which is about 250 times sweeter than sucrose. Other powerful natural sweeteners include sweet proteins such as thaumatin, found in the fruit of *Catemfe*, a plant native to West Africa. Egg lysozyme, an antibiotic protein found in chicken eggs, also exhibits sweetness." In this example, the context pruning system selects all contexts as relevant rather than (a) removing all contexts, (b) removing too many contexts, (c) selecting the wrong contexts, or (d) presenting false information as true. The first section, such as the title, is saved to provide background information for understanding the saved section.

[0100] In another example, the query "What was the Tower of London originally used for?" is received. A passage with the original context that is sufficiently similar to the query can be found. In the example, the full text is as follows:

[0101] "The Tower of London. In the 16th century, the Tower of London earned a notorious reputation as a gloomy and horrific prison. However, this wasn't always the case. As a royal palace, it was used by monarchs to imprison people for various reasons. However, it was mainly used to imprison people of higher status than ordinary citizens for short periods, as there were many other prisons that imprisoned citizens elsewhere. Contrary to the common image of the Tower of London, prisoners were able to purchase better food and amenities such as tapestries through the Tower of London caretaker, making their stay more comfortable. Originally, the Tower of London, like other castles, played a secondary role in housing prisoners, and there were no facilities specifically for prisoners until 1687, when a brick warehouse known as the "Soldiers' Prison" was built on the northwest side of the White Tower. The Tower of London's notoriety as a symbol of torture and imprisonment is thanks to 16th-century religious propagandists and 19th-century romantics." The context pruning system can leverage this passage, found using a general similarity filter, to determine a subset (if any) of the clauses within that passage that are relevant to the query. In this example, the context pruning system prunes the context with the following text:

[0102] "The Tower of London. This former royal palace was used by monarchs to imprison people for various reasons. However, it was primarily used to imprison people of higher status than ordinary citizens for short periods, as there were many other prisons for such people elsewhere." In this example, the context pruning system selects the single most relevant section in the passage rather than (a) removing all context, (b) removing too much context, (c) retaining too much context, (d) selecting the wrong context, or (e) presenting false information as true. The first section, such as the title, is preserved to provide background information for understanding the preserved section.

[0103] Pruning Multiple Passages In various examples, a general similarity filter can find multiple passages that are potentially relevant to the query, and the content pruning system can analyze these passages in parallel (e.g., using different content pruning system worker threads), together, in order of passage placement, or serially to determine the saved set of clauses (if any) for each passage. The saved clauses for each passage are combined as context to generate a context-based natural language response to the query and submitted with the query.

[0104] In one scenario, contexts from multiple passages can be ranked such that clauses from higher-ranking passages appear first in prompts to a natural language processing system, followed by clauses from other passages, with clauses grouped by passage. In another scenario, contexts from multiple passages can be cross-arranged so that higher-ranking clauses appear first, followed by lower-ranking clauses, regardless of which passage the clause belongs to. In yet another scenario, contexts from a high-ranking passage, the top five passages, or the top N passages can be combined and provided to a natural language processing system as context for generating query results.

[0105] In another scenario, the context of each passage can be used in a separate call to a natural language processing system to produce a different result. The individual results can be merged via another call to the natural language processing system to determine the most common or most comprehensive result among the individual results. In another embodiment, another call to the natural language processing system can use the individual results as a new context for a new query to the LLM that re-raises the original question. In yet another embodiment, the individual results of the natural language processing system can be merged using a machine learning model or other logic that senses the most common result in the result set, for example, based on which result has the vector embedding with the smallest total distance to the vector embeddings of the other results, without making another call to the natural language processing system.

[0106] • Adjusting the pruning level In one embodiment, the model used by the context pruning system outputs token-level predictions despite the clause-level labeling work. Despite the token-level predictions, the probability of including a token in the final context due to the clause-level goals used for training can naturally be concentrated at the clause level. However, in some cases, subclauses can be selected or preferred. To analyze the clause and decide whether to save it without the risk of losing meaning, the context pruning system can apply a "sentence rounding" or "clause rounding" procedure. For each clause, the context pruning system checks the ratio of saved tokens (predicted label = 1) to the total tokens, and if this ratio is higher than 0.5 (or other specified and adjustable threshold), it can select the entire clause; otherwise, it can delete the sentence.

[0107] A tunable threshold that determines whether to preserve or prune clauses can be learned to approximate many other computationally intensive pruning systems, such as those that use expensive, computationally intensive large-scale language models to determine which contexts to remove and which to preserve. The tunable threshold can be adjusted additionally or alternatively by the user to provide more context-intensive responses to queries that may be slower and further confuse the natural language processing system, or to provide less context-intensive responses to queries that are faster but may lack relevant details that would help answer some queries. In this example, lowering the threshold may result in more clauses being preserved and fewer clauses being pruned, while raising the threshold may result in fewer clauses being preserved and more clauses being pruned.

[0108] Figure 3 is an illustrative diagram of a user interface 300 that shows options for adjusting thresholds used to efficiently and robustly prune context for search extension generation. As shown, the query manager 302 can allow authenticated users 304 to log in to submit queries, view detailed information, or adjust configuration settings. The query configuration interface 306 includes a query input area 308. It can display an extended view 310 of the context and queries submitted to the LLM. The query configuration interface 306 can also display query responses 312 received from the LLM. In this example, the query configuration interface includes several options 314 for adjusting various context pruning thresholds that apply to various context sources. Various sources can be identified based on content type, server, database, topic, etc., and the adjustment thresholds specified in option 314 can be used by the context pruning system when pruning content in passages that match the source, type, topic, etc. In this way, users can control the pruning behavior for various types of context.

[0109] Without specifying a particular threshold, user feedback can be integrated to increase or decrease the threshold based on whether important context or details are missing (threshold too high), whether the response contains too much irrelevant information or is based on too little, or whether the response is too slow (threshold too low). Different thresholds can be learned for different types of queries based on user feedback, and the thresholds can be adjusted as various queries are received and processed. In one embodiment, if a user provides positive or negative feedback (or only flagged results) without specifying a reason, a large language model that provides the entire passage, the original query, and the original runtime as context can be used to understand why the feedback is positive or negative or why it was flagged, and the threshold can be updated based on the reasons understood by the large language model.

[0110] In various embodiments, a context pruning system can determine how many clauses from a set of passages are relevant enough to be included as additional context to a query, based on how closely sentences relate to the query and how much the clauses overlap with each other. A content pruning system can determine which passages and clauses are relevant for each query across a large available set of various passages, but such determinations can be adjusted by adjusting the threshold score at which clauses are considered relevant or irrelevant. For example, increasing the score allows for pruning more clauses and retaining fewer clauses that are more relevant to the query and less redundant. Alternatively, decreasing the score allows for pruning fewer clauses and retaining more clauses that are deemed relevant to the query but not redundant. In this way, a context pruning system has logic for determining how many clauses to retain or prune, and logic for adjusting how aggressively clauses are pruned, rather than requiring the retention or pruning of a specific number of clauses in passages that are potentially relevant to the query.

[0111] In one embodiment, in addition to adjusting the pruning threshold, or as an alternative, training data is provided to the context pruning system to determine whether the context has been over-pruned or under-pruned. Feedback can be provided to the model according to the context pruning performed, and this feedback can be used to adjust the aggressiveness of pruning and / or balance importance or relative weights that limit redundancy with relative weights that have a high degree of closeness to the query. For example, if query matching is rated as more important, clauses with a low degree of closeness to the query will be pruned more aggressively. As another example, if duplicate restriction is considered more important, clauses that are similar to other unpruned clauses will be pruned more aggressively. Feedback can be used to adjust the overall score assigned to each clause for pruning purposes, and the overall score can be determined based on the weights assigned to duplicate restriction, the weights assigned to query matching, the degree of duplicate to the clause considering other unpruned clauses, and the degree of match between the clause and the query.

[0112] In one embodiment, a context pruning system is trained on a large language model, such as GPT3, which receives queries and contexts and is prompted to use this large language model to determine the context portion most relevant to the query contained in the prompt. The large language model may consume considerable resources to determine the optimal context to select. A smaller model, such as T5-L or DeBERTa, can be trained with fewer parameters for the output of the context selected by the large language model for maintenance or pruning. In this case, the selection of the large language model is treated as a source of truth (significant resources), and the smaller model is trained to emulate the source of truth. For example, the smaller model can be trained to predict the output of the large language model based on a limited set of parameters, such as contexts and embedded data for queries input to the large language model for training.

[0113] In various embodiments, contextual pruning systems can automatically adjust the pruning ratio depending on the dataset (exemplifiedly, pruning from 50% to 80% of the nodes). When the pruning threshold T is adjusted, the contextual pruning system can still vary depending on the dataset, but more or fewer nodes tend to be pruned depending on whether T is adjusted higher or lower. Exemplary T values ​​were set to 0.1 and 0.5, providing variable pruning within the same T value. Users can adjust T according to the amount of pruning required for their dataset. Some models specify a desired compression ratio as a hyperparameter, but adjusting T is different from adjusting the compression ratio because adjusting T does not guarantee that a particular percentage of nodes will be pruned, not pruned, or pruned to a specific percentage for a given question-context pair. Instead, adjusting T adjusts how similar a node must be to the query for preservation, or conversely, how dissimilar a node must be to the query for pruning. The range of similarity scores varies from query to query, which can result in different proportions of clauses being pruned for the same T.

[0114] • Improve efficiency and accuracy through pruning by extraction without incurring additional round trips of generative abstraction. Various extractive approaches can remove clauses from text while preserving other clauses without incurring additional rounds of generative abstraction. These additional rounds of generative abstraction not only incur additional time and resource costs for evaluating large-scale language models, but also risk altering or losing the meaning of clauses due to the merging of clause concepts without case-specific guidance. Generating case-specific guidance is even more costly and carries the risk of altering or losing the meaning of the clauses themselves. Extractive approaches do not attempt to merge disparate clauses to generate new and concise representations of them, but instead preserve clauses through grammatical, formal, printing, or other minor or non-semantic adjustments to the text. The extracted clauses can then be provided to large-scale language models as an extended context for executing natural language queries.

[0115] In various embodiments, one or more clauses can be preserved, and the unique meaning reflected in multiple clauses can be preserved without damaging the context by removing relevant but less relevant clauses from among the multiple preserved clauses. While multiple clauses can be preserved, one or more other clauses can be removed from the text without diluting the meaning of the preserved clauses by attempting to generate a new abstract context using a large-scale language model. A method that attempts to generate a new abstract context describing clauses that can be removed from the context clause set can concatenate related clauses that do not provide exactly the same factual details in the same manner, resulting in clauses with diluted meaning. Such diluted meaning can provide a useless context that lacks important details of individual clauses, potentially causing the large-scale language model to malfunction.

[0116] • Compact architecture for identifying and pruning potentially relevant contexts DeBERTa models and other BERT-based models, or compact models with millions, tens of millions, or hundreds of millions of parameters rather than billions of parameters, can operate in environments of high throughput, low cost, low latency, and high efficiency. Many BERT-based models have 100M, 200M, or fewer parameters. Other models such as Mistral-7B, T5-XL, Alpaca-7B, Llama-7B, Llama-2-7B, or Llama-2-7B-chat have billions of parameters, which results in reduced throughput, increased cost, increased latency, and decreased efficiency. DeBERTa models are publicly available for a wide range of applications, and this model or other compact models can be used for contextual pruning according to the techniques described herein. For further details on DeBERTa models, see He, This can be seen in P., Liu, X., Gao, J., & Chen, W. (2021). DeBERTa: Decoding-enhanced BERT with Dis-entangled Attention, Proceedings of the International Conference on Learning Representations (ICLR) 2021. Retrieved from https: / / openreview.net / forum?id=XPZlaotutsD on December 7, 2024, the entire content is incorporated hereby for reference for any purpose as fully explicitly stated herein. DeBERTa integrates and improves the BERT and RoBERTa models. For further details on the BERT model, see Devlin, This can be found in J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of a deep bidirectional transformer for language understanding. Proceedings of the 2019 North American Association for Computational Languages ​​Annual Meeting: Human Language Technology, Vol. 1 (Long and Short Papers), 4171-4186. Retrieved from https: / / aclanthology.org / N19-1423 on December 7, 2024, and its entire content is incorporated hereby for reference for any purpose as fully explicitly stated herein. Additional information on the RoBERTa model can be found in Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, This can be seen in L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pre-training approach. arXiv preprint arXiv:1907.11692. Retrieved from https: / / arxiv.org / abs / 1907.11692 on December 7, 2024, its entire content is incorporated herein by reference for any purpose as fully expressly stated.

[0117] Compact models are distinguished from open models, which require a natural language processor to accept open text input and generate an open text response that can reflect all publicly available information. Such open models are unsuitable for high-throughput or low-cost scenarios because they result in significant additional costs, latency, low throughput, and low efficiency.

[0118] Using a compact or lightweight model allows Search Augmentation Generating (RAG) with context pruning to be used in scenarios involving numerous prompts to a large language model. In this scenario, each individual prompt and the logic used to generate it are expected to consume a limited amount of resources that can be shared among various workers generating prompts in parallel for different queries across the cloud infrastructure and potentially different users. A lightweight model allows for the generation of context-specific prompts, but without requiring additional round trips by the large language model. These prompts provide transactional responses that the large language model uses for synchronous work, where subsequent steps are performed based on the large language model's results. Such synchronous operation is not feasible in scenarios requiring multiple data-intensive round-trip calls to the large language model, and multiple round trips per query can be prohibitively costly due to the resources consumed by the large language model with each round trip. Using a compact or lightweight model allows for context-specific queries that are performed without context, avoiding additional round trips by the large language model for various reasons.

[0119] As shown in Table 1 below, using a compact model can improve the compression time and MFLOPS used by contextual pruning systems compared to other pruning methods. Pruners such as LongLLMLingua, which use the LLama-2-7B-chat model, were the slowest in terms of contextual pruning speed. RECOMP abstract models use fewer MFLOPS for pruning, but are actually even slower due to their autoregressive properties. RECOMP extracting models produce a fixed number or ratio of pruned nodes and are not adaptive. In the case of combined models (contextual pruning systems with reranking capabilities), pruning is almost free.

[0120] [Table 1]

[0121] Table 2 below shows the speed improvement effect of compressing a downstream large-scale language model using a context pruning system (exemplary compression ratio of 50%). All executions were performed on a single Tesla V100-SXM2-32GB GPU equipped with vllm10. When the batch size was large, the exemplary context pruning system resulted in approximately a twofold speed improvement during inference, but even with a small batch size, the improvement in inference speed was still considerable.

[0122] [Table 2]

[0123] In various embodiments, the roberta-base (125M parameters) and / or roberta-large (355M parameters) models can be used, with some differences in compression ratio and / or efficiency.

[0124] A context pruning system is a robust, adaptable, and efficient context pruner for query responses, which can be integrated into a single model with reranking capabilities or offered as a lightweight, standalone model. The context pruning system can dynamically sense pruning ratios applicable to a given context and can be used across various QA domains. In various examples, the context pruning system accurately senses the number of relevant statements within a context, which are present in all positions and have contexts of varying lengths. Some embodiments focus on QA applications that process only one passage at a time, while others select potentially included passages before processing the clauses of the passage for ranking and pruning.

[0125] • Contextual pruning across domains In various embodiments, context pruning can be performed across multiple domains and multiple use cases to guide large-scale language models or other natural language query processing systems. The context remaining after pruning clauses from the initial context can be used to request information relevant to various topics from the query processing system using various domain-specific contextual passages, and can perform various tasks or generate content suitable for various models specialized to handle various result templates, patterns, or different types of questions. The initial query can be communicated to a special-purpose model by analyzing the initial query and its contained context, which may have already been pruned according to the techniques described herein. The analysis can match the query and contained context with a set of exemplary queries, exemplary responses, educational materials, metadata, or other information relevant to various models for answering various types of questions. For example, the query and / or context can be matched to the nearest model using the cosine distance between information about the query and / or context and examples, metadata, or other information about the model.

[0126] In various embodiments, training can be performed on NQ (84K queries) and / or MS MARCO downsampled to the same size or full MS MARCO (370k queries). Using larger training datasets improves results but consumes more resources. Training on NQ data can degrade re-ranking performance because NQ data is not a common data type for training rankers again. Supplementing training data with multi-domain datasets can improve results.

[0127] Concurrent ranking and pruning of contexts In one embodiment, reranking and contextual pruning are performed together and simultaneously in the Search Enhancement Generation (RAG) pipeline, rather than being treated as separate steps. Since reranking and pruning can be performed in a single forward step, the computational overhead of contextual pruning is eliminated, and the pruning step can be performed almost "freely" if the reranking step is included in the process, regardless of whether pruning is involved or not.

[0128] Contextual pruning systems can be implemented in various embodiments. In a standalone embodiment, the passage selection model, the passage reranking model, and the clause selection model are trained individually. In a reranking embodiment, the passage reranking model and the clause selection model are trained together. In other words, the reranking embodiment uses one model with two different predictive heads: one for passage reranking and the other for clause selection.

[0129] In one embodiment, the same model used to determine the binary mask for each token in a clause is also used to determine the clause-level total similarity score and reranking score for a passage containing a saved clause. Since the token-level score and the clause-level score are determined using the same model, clauses can be processed simultaneously in a single pass via a binary mask output that includes both the token-level and clause-level scores. The clause-level score can be used to determine the passage-level reranking score, which determines the rank or order of passages between passages with different scores.

[0130] In other embodiments, the first model is used for reranking and the second model for pruning. This embodiment is more flexible to various rerankers but has the disadvantage of requiring separate reranking and pruning steps. Assuming that reranking is performed before pruning, pruning can benefit from the additional information generated by the reranker and can wait until that information is provided. If pruning is performed before reranking, reranking does not need to wait for the reranking information and does not benefit from that information. Combining reranking and pruning in the same step that is determined algorithmically simultaneously allows the benefits of reranking information to be retained without the additional cost incurred by performing pruning in a separate step.

[0131] In one embodiment, a combination of a reranker and a pruner can be trained to mimic another reranker to determine a similar reranking score. Once the combined reranker achieves a similar reranking score, the other reranker can be deactivated in the loop, and the combined reranker will then be responsible for reranking.

[0132] A cross-encoder reranker can share the same architecture and input (question-passage pairs) as a context-pruning system. The context-pruning task (partial selection of context useful for generating answers to questions) can be performed using similar logic to reranking (estimating the relevance of context in relation to a question), and embodiments of the reranker in a context-pruning system leverage knowledge transfer between these two related tasks by integrating the two approaches into a single model. The reranking head of the model outputs scalar predictions for [BOS] tokens, and the pruning head of the model outputs token-specific predictions for passage tokens. Such outputs can be combined into the same binary object and determined together section by section. The context-pruning system can fine-tune a pre-trained reranker for specific labeling targets and add a rank "normalizer" to maintain initial reranking capabilities. In one embodiment, the normalizer is the mean squared error loss over the reranking score of the initial reranker. This can be viewed as a point-by-point score distillation setting for which the initial model is the teacher. In one example, the final loss function is defined as follows:

[0133]

number

[0134]

number

[0135] Here, N is the number of data points (query-passage pairs), and x n is L n+1 It is a sequence of input tokens (queries, passages, and BOS concatenated at position 0), z n This is L output from the model. n+1 These are the number of prediction sequences, y nL for context pruning n A sequence of target binary labels, where sn is the teacher score (initial re-ranking), and z n,0 This is the ranking score predicted using the BOS representation.

[0136] • Triggering downstream activity as a result of context-based natural language processing Natural language queries, such as prompts to large-scale language models, can be used to generate output that answers the query. When pruned context is provided along with a natural language query, a natural language query processor or large-scale language model can use the pruned context to provide context-based answers to the query without requiring the large-scale language model to be separately trained to consider the specific context. The answer to the query can generate text such as a short response that considers the facts provided in the context, or executable code that considers the code libraries, APIs, or preferred protocols provided in the context. Such answers can be used to display information on a screen, to enable a computer or robot to read information aloud using a speech-to-text model, to enable a computer or robot to perform actions such as physical actions or electronic communications, or to provide input to other models trained for a specific purpose. Context can inform the large-scale language model of a private or limited problem space, provide knowledge about a specific environment, machine operation or problem-solving, or other technology, or provide other expertise applicable to a specific domain.

[0137] In various embodiments, the response to a query can generate text, such as a short or long answer, that describes facts or details provided in the context. The answers and details described may be work-related, user-related, or domain-related. The response to a query can generate executable code that describes a code library, API, or preferred protocol provided in the context. The response to a query can generate structured data, such as a JSON (JavaScript Object Notation) object, or other data format that describes data elements provided in the context. The response can be used for any purpose in all downstream applications, including, but not limited to, work triggers, displaying relevant information to the user, triggering the activation of equipment by executable code received with the response or other instructions, and / or prompting for a large language model or interacting with other APIs for additional information.

[0138] Computer System Architecture Figure 4 shows a distributed system 400 for efficient and robust contextual pruning for search extension generation.

[0139] As shown in the diagram, client computing devices 432, 434, 436, 438, and / or 440 communicate with server instances 402 and / or 404 via one or more communication networks 430. Users can use the client computing devices to access services provided by server instances 402 and / or 404. Client computing devices may include smartphones or other portable handheld devices, general-purpose computers such as personal computers and laptops, workstation computers, personal assistant devices, wearable devices, device firmware, game systems, and the like.

[0140] Server instance 402 includes functional components 406, 408, 410, and 412 that provide functions to support the services provided by server instance 402. For example, functions can be provided through web pages or applications accessible via client devices 432, 434, 436, 438, and / or 440. Functions can retrieve data, perform calculations on data, sort, filter, or select data, and display information to client devices 432, 434, 436, 438, and / or 440. Server instance 402 can store and retrieve data to support services provided in interaction with data repositories 422 and 424.

[0141] Server instance 404 includes functional components 414, 416, 418, and 420 that provide functionality to support the services provided by server instance 404. For example, this functionality can be provided through a web page or application accessible via client devices 432, 434, 436, 438, and / or 440. This functionality can retrieve data, perform calculations on data, sort, filter, or select data, and display information to client devices 432, 434, 436, 438, and / or 440. Server instance 404 can store and retrieve data to support the services provided by interacting with data repositories 426 and 428. Server instances 402 and 404 can also communicate with each other to support the services provided, and can leverage the functionality provided by the functional components of other server instances to provide more comprehensive services to client devices 432, 434, 436, 438, and / or 440.

[0142] Such client and server devices can use a variety of operating systems, including Microsoft Windows®, Apple MacOS®, UNIX® or UNIX-like operating systems, Linux® or Linux-like operating systems, Android®, and Apple iOS®, which provide software access to basic hardware resources.

[0143] Data repositories 422, 424, 426, and 428 store data that may include instructions, saved objects, server instance activity logs, and other data. Server instances 402 and 404 may be computer systems as described in Figure 5, respectively, and may be virtually hosted on computer systems that provide other functions, or other server instances may be hosted using the shared and non-shared resources of the host computer system.

[0144] Figure 5 is an illustrative diagram showing a computer system 500 that can be used to efficiently and robustly prune context for search extension generation.

[0145] As shown in the figures, the computer system 500 includes a storage subsystem 508 which may include storage media 510, 514, 518, and 520. Storage media 510 may, for example, include stored instructions for executing an application or server function. Storage media 514 may include system memory 516 for managing objects operated by the computer system 500 during application execution or service provision. The storage subsystem 508 may include one or more non-transient computer-readable storage media, which may be in the form of a mobile and tangible computer program product or non-mobile and tangible storage hardware, and, when executed by one or more data processors such as the processing unit 522, the special-purpose processing unit 536, or the computer systems 502, 504, or 506, store instructions causing one or more data processors to perform some or all of the methods disclosed herein. The term "computer-readable medium" as used herein does not include transient electrical or electromagnetic signals (e.g., carrier waves) propagated through the medium. Therefore, the term "computer-readable medium" can be considered tangible and non-transient. A non-temporary computer-readable medium may be a volatile medium that is usable when power is supplied to the computer system 500 or the medium itself, or a non-volatile storage medium in which the information stored therein can be maintained even when the power to the computer system or the medium itself is turned off.

[0146] The processing unit 522 may include processing units 524, 526, 528, 530, 532, and 534. The processing units may operate individually or independently, or they may work together to perform complex tasks using parallel processing of sub-tasks that should be performed as part of the complex task. As the work is performed, the processing units may store information in the storage subsystem 508.

[0147] The special-purpose processing unit 536 may include a special-purpose processing unit 538, such as a processing unit optimized for performing vector-based calculations to support machine learning tasks. The special-purpose processing unit 536 also includes a graphics processing unit 540 that communicates with processing unit 522 and storage subsystem 508 to display information on display 542.

[0148] The communication device 544 may include a WiFi device 546, an Ethernet device 548, and / or a short-range wireless device 550. The WiFi device 546 and the Ethernet device 548 may be part of a network device for communicating with other devices such as computer systems 502, 504, and 506 over the Internet using a set of communication protocols such as Transmit Control Protocol / Internet Protocol (TCP / IP). The short-range wireless device may also communicate with computer systems 502, 504, and 506 using short-range protocols such as Bluetooth®, Z-Wave®, and Zigbee®.

[0149] The computer system 500 may include input devices such as a mouse, keyboard, touchscreen, touchpad, buttons, and joystick, and output devices such as a display 542 and speakers.

[0150] The apparatus and method described in this application can be configured to perform one or more specific functions implemented in a computer program, and can be partially or entirely implemented by a special-purpose computer generated. The functional blocks, flowchart components, and other elements described above are used as software specifications, which can be converted into a computer program by a skilled technician or programmer. The computer program includes processor-executable instructions stored in at least one non-temporary, tangible, computer-readable medium. The computer program may also include, or rely on, stored data.

[0151] Computer programs can include: (i) descriptive or hierarchical text to be parsed, such as HTML (hypertext markup language), XML (extensible markup language), and JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code executed by an interpreter; and (v) source code compiled and executed by a just-in-time compiler. As an example, source code can be created using the syntax of languages ​​including C, C++, C#, Objective C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP (Hypertext Preprocessor), Scale, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.

[0152] While specific embodiments have been described, various modifications, changes, alternative configurations, and equivalents are possible. Embodiments are not limited to operation within a specific data processing environment but can freely operate within multiple data processing environments. Furthermore, while specific embodiments have been described using a specific set of transactions and steps, those skilled in the art will understand that this is not intended to be restrictive. Some flowcharts describe operations as sequential processes, but many operations can be performed in parallel or concurrently (i.e., operations occurring within the same time frame can potentially overlap but are not necessarily exactly concurrent. For example, a system configured to execute multiple units of work (e.g., threads) concurrently using multiple physical processors may allow parallel execution of multiple processes, each capable of executing one or more threads, or a single (multithreaded) process may be used to divide the unit of work into multiple smaller threads that can execute concurrently within a single process), and the order of operations can be reordered. The terms “First,” “Second,” “Third,” “Fourth,” “Fifth,” and “Sixth” are naming conventions used to refer to distinct items individually and are not intended to convey a specific order or priority unless such order or priority is communicated in another language. For example, unless otherwise specified, "Section 1" does not necessarily refer to the first section in the passage; the name "Section 1" can be used to distinguish it from "Section 2," which is not necessarily the second section in the passage, and "Section 3," which is not necessarily the third section in the passage. The process may have additional steps not included in the drawings. The various features and embodiments of the above-described aspects can be used individually or in combination.

[0153] Furthermore, while certain embodiments have been described using specific hardware and software combinations, it should be recognized that other hardware and software combinations are also possible. Certain embodiments can be implemented using hardware alone, software alone, or a combination of both. The various processes described herein can be implemented using the same processor or different processors in any combination.

[0154] When a device, system, component, or module is described as being configured to perform a particular operation or function, such configuration can be achieved, for example, by designing an electronic circuit to perform the operation, or by programming a programmable electronic circuit (e.g., a microprocessor) to perform the operation (e.g., by executing computer instructions or code), or by programming a processor or core to execute code or instructions stored in a non-temporary storage medium, or by any combination thereof. Processes can communicate using a variety of techniques, including but not limited to existing techniques for inter-process communication, and pairs of different processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0155] This specification provides specific details to give a complete understanding of embodiments of the invention. However, embodiments can be carried out without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary details so as not to obscure the embodiments. This description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of other embodiments. Rather, the description of such embodiments can provide a practicable description for those skilled in the art to realize various embodiments. Various modifications can be made to the function and arrangement of the elements.

[0156] Therefore, this specification and the drawings should be considered illustrative, not restrictive. However, it may be apparent that additions, deletions, omissions, and other modifications and alterations may be made without departing from the broader spirit and scope expressed in the claims. Thus, while specific embodiments have been described, this is not restrictive. Various modifications and equivalents are within the scope of the following claims.

Claims

1. A method for realizing a computer, Steps to access the query, A step of determining one or more passages that are potentially relevant to the query from a content corpus containing multiple clauses from multiple passages, wherein the one or more passages contain a subset of three or more clauses. A step of encoding a subset of three or more sections in order to determine a section-query relevance score for each section, wherein the section-query relevance score of a particular section of at least one of the subsets of three or more sections is determined by the content of at least one other section of the subset; A step of generating a modified subset of clauses by removing one or more clauses from the subset of three or more clauses, based at least partially on the clause-query relevance scores of one or more clauses from the subset of three or more clauses; A step of generating a prompt for a natural language processor, the prompt comprising an instruction at least partially based on the query and at least a portion of the modified clause as a context for generating a response by the natural language processor, The steps of prompting the natural language processor with the prompt, A computer implementation method comprising the steps of accessing and storing the result generated by the natural language processor at least in part based on executing the prompt.

2. The step of encoding a subset of the three or more clauses is: For each of the three or more subsets of the above-mentioned clauses: The steps include determining the token-query relevance score of each token in the preceding section based at least partially on the token-query similarity between the token embedding of the token and the query embedding of the query, The step of determining the section-query relevance score of the section based at least in part on the token-query relevance score of each token in the section, The token embedding for at least one token in at least one of the aforementioned sections is, A computer implementation method according to claim 1, which is at least partially based on the content of at least one section and at least one other section of a subset of the three or more sections.

3. The step of determining one or more passages from the content corpus is: The computer implementation method according to claim 1, comprising the step of determining the passage-query similarity between the vector embeddings of the plurality of passages and the vector embedding of the query.

4. The steps include dividing multiple documents into multiple passages containing consecutive sentences, The method further includes the steps of training a model to identify which of the aforementioned multiple passages is most relevant to each query, and generating a trained passage selection model, The step of determining one or more passages is: The computer implementation method according to claim 1, using the trained passage selection model.

5. At least partially, Prompting a large language model with multiple prompts requesting the selection of the most relevant passage, By assigning weights to the features of a node based at least partially on the results of the prompts in the aforementioned multiple prompts, The process further includes the step of training a model to identify which of a subset of clauses is most relevant to each query, and generating a trained clause selection model. The step of encoding a subset of the three or more clauses is: The computer implementation method according to claim 1, using the aforementioned trained node selection model.

6. The aforementioned trained node selection model is The computer implementation of claim 5, which is trained to select the subset of clauses that has the highest relevance to each query, in relation to the total amount of information expressed in each subset of the clauses for a particular passage.

7. The step of determining one or more passages is: Based at least partially on one or more passage-query similarity scores determined using an initial passage selection model, The aforementioned computer implementation method is The steps include adjusting the one or more passage-query similarity scores using other passage selection models, The method further includes the step of determining the one or more clause-query relevance scores for the one or more clauses using a clause selection model, The aforementioned section selection model is, Apart from the aforementioned other passage selection models trained to determine the most relevant passages, The aforementioned other passage selection models are: The computer implementation method according to claim 1, which integrates even more features than the initial passage selection model described above.

8. The step of determining one or more passages is: Based at least partially on one or more passage-query similarity scores, The aforementioned computer implementation method is The process further includes determining the one or more section-query relevance scores for the one or more sections and adjusting the one or more passage-query similarity scores using a combined machine learning model, The step of removing one or more clauses from the subset of three or more clauses and generating the modified subset of clauses is: The computer implementation method according to claim 1, which is at least partially based on the one or more clause-query relevance scores.

9. The step of determining one or more passages from the content corpus is: Calculate the initial rank of two or more sections, The aforementioned computer implementation method is The process further includes the step of simultaneously reranking the two or more passages and selecting one or more clauses to remove, The computer implementation method according to claim 1, wherein the natural language processor includes a large-scale language model.

10. The computer implementation method according to claim 9, further comprising the step of selecting one or more sections to remove based on the subset of sections that is most relevant to the query compared to the total amount of information expressed in each subset of the section subsets for a particular passage.

11. A step to display an option to adjust the threshold for selecting the nodes to remove, The method further includes receiving the selection of the option to adjust the threshold and deriving the adjusted threshold, The one or more of the above sections, The computer implementation method according to claim 2, selected for removal based at least partially on the adjusted threshold.

12. The computer implementation method according to claim 1, wherein the first section of the subset of three or more sections is the title of the first section of the plurality of passages, the second section of the subset of three or more sections is in the body of the first section of the plurality of passages, the third section of the subset of three or more sections is also in the body of the first section of the plurality of passages, and the third section is removed from the subset of three or more sections to generate the modified subset of sections.

13. It is a system, One or more processors, The system includes one or more non-temporary computer-readable media that store instructions causing the system to perform the following set of actions when executed by the system, The set of operations is, Accessing queries and Determining one or more passages potentially relevant to the query from a content corpus containing multiple clauses from multiple passages, wherein the one or more passages contain a subset of three or more clauses. Encoding a subset of three or more sections in order to determine a section-query relevance score for each section, wherein the section-query relevance score of at least one section in the subset of three or more sections is determined by the content of at least one other section in the subset of three or more sections. Based at least partially on the clause-query relevance score of one or more clauses from the subset of three or more clauses, remove one or more clauses from the subset of three or more clauses to generate a modified subset of clauses. To generate a prompt for a natural language processor, wherein the prompt includes at least a portion of the modified clause as a command at least partially based on the query and a context for generating a response by the natural language processor, The natural language processor is prompted by the prompt, A system comprising accessing and storing results generated at least in part based on the natural language processor executing the prompt.

14. Encoding a subset of the three or more clauses is: For each of the three or more subsets of the above-mentioned clauses: The token-query relevance score for each token in the preceding section is determined at least in part on the token-query similarity between the token embedding of the token and the query embedding of the query, This includes determining the section-query relevance score within the section based at least in part on the token-query relevance score of each token in the section, The token embedding for at least one token in at least one of the aforementioned sections is, The system according to claim 13, which is at least partially based on the content of at least one section and at least one other section of a subset of the three or more sections.

15. The set of operations described above is at least partially: Prompting a large language model with multiple prompts requesting the selection of the most relevant clause, By assigning weights to the features of a node based at least partially on the results of the prompts in the aforementioned multiple prompts, This further includes training a model to identify which of a subset of clauses is most relevant to each query, and generating a trained clause selection model. Encoding a subset of the three or more clauses is: The system according to claim 13, using the aforementioned trained node selection model.

16. Determining one or more of the aforementioned passages is Based at least partially on one or more passage-query similarity scores determined using an initial passage selection model, The set of operations is, Adjusting the one or more passage-query similarity scores using other passage selection models, The method further includes determining the one or more clause-query relevance scores for the one or more clauses using a clause selection model, The aforementioned section selection model is, Apart from the aforementioned other passage selection models trained to determine the most relevant passages, The aforementioned other passage selection models are: The system according to claim 13, which integrates even more features than the initial passage selection model described above.

17. Determining one or more of the aforementioned passages is Based at least partially on one or more passage-query similarity scores, The set of operations is, The method further includes determining the one or more section-query relevance scores for the one or more sections and adjusting the one or more passage-query similarity scores using a combined machine learning model, Removing one or more clauses from the subset of three or more clauses and generating the modified subset of clauses is, The system according to claim 13, which is at least partially based on the one or more clause-query relevance scores.

18. Determining one or more passages from the aforementioned content corpus is, Calculate the initial rank of two or more passages, The set of operations is, The method further includes using a single training model to simultaneously rerank the two or more passages and select one or more clauses to remove, The system according to claim 13, wherein the natural language processor includes a large-scale language model.

19. The set of operations is, Display an option to adjust the threshold for selecting the nodes to remove, The process further includes receiving the selection of the option to adjust the threshold and deriving the adjusted threshold, The system according to claim 13, wherein one or more of the aforementioned sections are selected for removal based at least partially on the adjusted threshold.

20. One or more non-temporary computer-readable media that store instructions for performing the following set of actions during execution, The set of operations is, Accessing queries, Determining one or more passages potentially relevant to the query from a content corpus containing multiple clauses from multiple passages, wherein the one or more passages contain a subset of three or more clauses. Encoding a subset of three or more sections to determine a section-query relevance score for each section, wherein the specific section-query relevance score of at least one section in the subset of three or more sections is determined by the content of at least one other section in the subset of three or more sections. Removing one or more clauses from the subset of three or more clauses to generate a modified subset of clauses, based at least partially on the clause-query relevance score of one or more clauses from the subset of three or more clauses. To generate a prompt for a natural language processor, wherein the prompt includes an instruction at least partially based on the query, and at least one of the modified clauses as a context for generating a response by the natural language processor. Prompting the natural language processor with the prompt, and One or more non-temporary computer-readable media, including accessing and storing results generated at least in part based on the natural language processor executing the prompt.

21. Encoding a subset of the three or more clauses is: For each of the three or more subsets of the above-mentioned clauses: The token-query relevance score for each token in the preceding section is determined at least in part on the token-query similarity between the token's token embedding and the query's query embedding, and The procedure includes determining the section-query relevance score of the preceding section based at least in part on the token-query relevance score of each token in the preceding section, The token embedding for at least one token in at least one of the aforementioned sections is, One or more non-temporary computer-readable media according to claim 20, based at least in part on the content of at least one section and at least one other section of a subset of the three or more sections.

22. The set of operations described above is at least partially: Prompting a large language model with multiple prompts requesting the selection of the most relevant clause, By assigning weights to the features of a node based at least partially on the results of the prompts in the aforementioned multiple prompts, This further includes training a model to identify which of the subsets of clauses is most relevant to each query, and generating a trained clause selection model. Encoding a subset of the three or more clauses is: One or more non-temporary computer-readable media according to claim 20, using the aforementioned trained node selection model.

23. Determining one or more of the aforementioned passages is Based at least partially on one or more passage-query similarity scores determined using an initial passage selection model, The set of operations is, Adjusting the one or more passage-query similarity scores using other passage selection models, and The method further includes determining the one or more clause-query relevance scores for the one or more clauses using a clause selection model, The aforementioned section selection model is, Apart from the aforementioned other passage selection models trained to determine the most relevant passages, The aforementioned other passage selection models are: One or more non-temporary computer-readable media according to claim 20, which integrates more features than the initial passage selection model described above.

24. Determining one or more of the aforementioned passages is Based at least partially on one or more passage-query similarity scores, The set of operations is, The method further includes determining the one or more section-query relevance scores for the one or more sections and adjusting the one or more passage-query similarity scores using a combined machine learning model, Removing one or more clauses from the subset of three or more clauses and generating the modified subset of clauses is, One or more non-temporary computer-readable media according to claim 20, based at least in part on the one or more clause-query relevance scores.

25. Determining one or more passages from the aforementioned content corpus is, Calculate the initial rank for two or more passages, The set of operations is, The method further includes simultaneously reranking the two or more passages using a single training model and selecting one or more clauses to remove, The natural language processor comprises one or more non-temporary computer-readable media according to claim 20, including a large-scale language model.

26. The set of operations is, Display an option to adjust the threshold for selecting the nodes to remove, and The process further includes receiving the selection of the option to adjust the threshold and deriving the adjusted threshold, The one or more non-temporary computer-readable media according to claim 20, wherein the one or more sections are selected for removal based at least in part on the adjusted threshold.

27. A computer implementation method that optimizes sections retrieved from a data store and determines the answer to a query received from a user, The aforementioned computer implementation method is The steps include: searching the data store for a set of first clauses ranked by their relevance to the answers to the query; Process the aforementioned query and the first set of clauses simultaneously: The first set of clauses is pruned to generate a subset of the first set of clauses, A step of generating a subset of clauses in a context pruning subsystem, which generates a reranking score for at least one passage containing the subset of the first set of clauses based on its relevance to the answers to the query, The steps include: prompting a natural language processor to provide the answer to the query, the prompting step using the query and a subset of the clauses as context provided for providing the answer to the query; The context pruning subsystem is trained to prune at least some of the clauses using a cross-encoder that encodes the queries and candidate clauses of the training set and generates the following output: A remaining subset of the candidate clauses for the query in the training set, wherein the remaining subset excludes at least some of the pruned clauses, and A computer implementation method, including ranking the remaining subset of the candidate clauses.

28. The steps include receiving the query from the user interface, The step of outputting the answer to the user interface is included, At least some of the aforementioned sections are pruned using a binary mask, The ranking of the remaining subset of the candidate clauses is generated using the reranking score. The computer implementation method according to claim 27, wherein the context pruning subsystem is stored in memory and the natural language processor is a large-scale language model.

29. The aforementioned user interface is Includes executable logic for receiving input via an interactive input device, The aforementioned interactive input device is A computer implementation method according to claim 28, comprising a display, keyboard, mouse, microphone, speaker, or a combination thereof.

Images