Selective and adaptive retrieval-augmented generation with context compression
The SARA framework addresses LLM limitations by using context compression vectors to efficiently provide relevant contextual information to LLMs, enhancing performance and reducing token consumption without retraining, thus improving answer relevance and accuracy.
Patent Information
- Application Number
- PCT/US2026/027899
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-13
- Filing Date
- 2026-05-12
- Publication Date
- 2026-11-19
AI Technical Summary
Large language models (LLMs) face limitations in processing long input prompts due to quadratic scaling of transformer models, leading to constrained context windows and inefficient retrieval-augmented generation (RAG) systems that struggle with providing sufficient relevant contextual information, resulting in inaccurate and resource-intensive retraining.
A unified and efficient RAG framework, called SARA, provides a combination of compressed and uncompressed contextual information to LLMs, using context compression vectors to reduce token consumption while maintaining relevance and accuracy by iteratively selecting and ranking context data elements.
SARA enhances LLM performance by improving answer relevance, correctness, and semantic similarity, reducing token consumption, and avoiding costly retraining, while being applicable across various LLM backbones and domains.
Smart Images

Figure US2026027899_19112026_PF_FP_ABST
Abstract
Description
PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001SELECTIVE AND ADAPTIVE RETRIEVAL- AUGMENTED GENERATION WITH CONTEXT COMPRESSIONCROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This is a PCT application that claims priority' to U. S. Provisional Application No.63 / 805,169, filed on May 13, 2025, which is herein incorporated by reference.BACKGROUND
[0002] Recently, generative Al models, including large language models (LLMs), have seen increasing use in a variety of applications, and have shown strong capabilities across various natural language understanding and generation tasks (Xiao et al., 2024a, Zhao et al., 2024). LLMs can generally convincingly produce natural language outputs (“responses”) to natural language inputs (e.g., “queries” or “prompts”). Thus, LLMs have seen increasing use as chatbots and personal assistants. As an example, a user can query an LLM using a prompt such as “What is the population of the Paris metropolitan area?”. The prompt can be converted into a sequence of “tokens”, e g., vectors or numerical representations of individual words, collections of letters (such as the suffix “-ing”), punctuation marks, etc. The LLM can process the sequence of tokens and produce its own token sequence, which can then be decoded to produce a response to the user’s query. The response, e.g., “The population of the Paris metropolitan area is 13 million people” can then be delivered to the user.
[0003] LLMs are parametric in nature, meaning that any LLM response is dictated by the parameters of the LLM, and such responses are inherently constrained by the scope, domain, and recency of their training data (Jin et al., 2024b; Liu et al., 2025). Thus an LLM may not accurately respond to a given query, e.g., reporting a Paris metropolitan area population that is out of date or erroneous (e.g., “hallucinated”). Further, while LLMs may generate reasonable and accurate responses for general purpose or common interest queries, they may perform poorly on queries relating to specialized subjects, as they may have been trained on comparatively little training data for those specialized subjects.
[0004] A solution to this problem is to train LLMs on large diverse datasets that include data relating to any specialized field that may be relevant to a particular LLM application, and additionally periodically retrain LLMs with the most recent data. However, training and1KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 retraining LLMs is resource intensive. As such, data scientists and Al experts research other methods to improve LLM performance, including retrieval-augmented generation (RAG) (Lewis et al., 2020), which generally relates to processes for augmenting prompts using data retrieved from external non-parametric knowledge sources, e.g., databases or other data sources.
[0005] However, there are still several problems with current RAG systems, resulting in poor LLM performance. Embodiments address these and other problems, individually and collectively.SUMMARY
[0006] This Summary is provided to introduce a selection of concepts in a simplified form that are further described herein in the Detailed Description. This Summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
[0007] Embodiments of the present disclosure are directed to various methods and systems associated with a unified and efficient retrieval-augmented generation (RAG) framework. Such methods and systems can be practiced with any applicable foundation model but are particularly applicable for use with large language models (LLMs). Methods according to embodiments can be performed by computer systems, including computer systems that instantiate and run foundation models (e.g., perform LLM inference), as well as computer systems that interface with external model provider servers (e.g., associated with LLM providers or other ‘‘machine learning as a service” (MLaaS) providers) for the purpose of performing inference.
[0008] Generally, RAG enhances LLM performance via data retrieved from an external datastore. As an example, upon receiving a prompt such as '‘What is the population of the Paris metropolitan area?”, a RAG system can query a database using e.g., a keyword search, and identify' a current demographics report document for Paris. This document could then be incorporated into a prompt to an LLM. e.g., “in view of the following context: <inserted demographics report document content>, what is the population of the Paris metropolitan area?” By providing an LLM with information that is relevant, recent, and accurate, a RAG system improves the performance of an LLM model w ithout requiring costly retraining.2KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0009] Unfortunately, due to the nature of LLMs models and underly ing transformer model architectures, LLM processing scales quadratically with the number of input tokens, which is proportional to the length of provided prompts. As a consequence, LLMs have limited ■‘context windows”, which prevent them from accepting and processing arbitrarily large input prompts. This has further consequences for RAG systems, which are thus limited in the amount of additional contextual information that can be provided to LLMs, limiting their usefulness.
[0010] Embodiments can address this problem by providing RAG methods and systems that provide a combination of compressed and uncompressed contextual information, in ranked order, to a foundation model (e.g., an LLM). In this way, the most relevant contextual information (e.g., based on the ranking) can be provided to the foundation model in an uncompressed form, while less relevant (but still relevant) information can be provided in a compressed form, thereby reducing token consumption and avoiding the problems described above. Embodiments address various other existing problems with RAG systems and provide various advantages, which are described in the Detailed Description below.
[0011] In more detail, one embodiment is directed to a method for generating a query response corresponding to a query using a foundation model and a plurality of context data elements. The method can be performed by a computer system. The computer system can identify a plurality of context data elements based on relevance to the query. The computer system can generate a plurality of context compression vectors corresponding to the plurality of context data elements. The computer system can determine one or more primary’ context data elements of the plurality of context data elements, a determined order of the one or more primary context data elements, and one or more secondary context data elements of the plurality of context data elements. These can be determined based on the plurality of context compression vectors. The determined order can be based on relevance of each of the one or more primary context data elements to the query. The computer system can generate a prompt comprising the one or more primary context data elements, one or more compression vectors corresponding to the one or more secondary context data elements, and the query7. The one or more primary context data elements can be ordered in the prompt based on the determined order. The computer system can generate a query response using the prompt and the foundation model.3KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0012] Another embodiment is directed to another method for generating a query response corresponding to a query using a foundation model and a plurality of context data elements. The method can be performed by a computer system. The computer system can identify a plurality of context data elements based on relevance to the query. The computer system can identify a first context data element from the plurality of context data elements based on a relation of the first context data element to the query. The computer system can generate a primary context data element set initially comprising the first context data element. The computer system can perform one or more iterative rounds. Each iterative round can comprise updating the primary context data element set to include a selected context data element from one or more remaining context data elements of the plurality of context data elements, thereby generating an updated primary’ context data element set. Each different selected context data element can be selected based on a comparison of each remaining context data element of the one or more remaining context data elements to the primary' context data element set. The computer system can generate a prompt comprising the query and a plurality of primary context data elements in the primary context data element set. The computer system can generate the query response using the prompt and the foundation model.
[0013] In addition to the methods described above (and other methods), some embodiments are directed to computer systems or other devices that can be configured to perform the methods described above or other methods. For example, one embodiment is directed to a computer system comprising one or more processors and a non-transitory computer readable medium coupled to the one or more processors. The non-transitory computer readable medium can comprise instructions that, when executed by the one or more processors, cause the computer system to perform the methods described above (or other methods described in the detailed description below).
[0014] A better understanding of the nature, additional features, advantages, and structure and operation of embodiments of the present disclosure may be gained with reference to the following detailed description, accompanying drawings, and the claims. In the accompanying drawings, like reference numbers can indicate identical or functionally similar elements.4KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001TERMS
[0015] A “server computer” may refer to a computer or cluster of computers. A server computer may be a powerful computing system, such as a large mainframe. Server computers can also include minicomputer clusters or a group of servers functioning as a unit. In one example, a server computer can include a database server coupled to a web server. A server computer may comprise one or more computational apparatuses and may use any of a variety of computing structures, arrangements, and compilations for servicing requests from one or more client computers.
[0016] A “client computer” may refer to a computer or cluster of computers that receives some service from a server computer (or another computing system). The client computer may access this service via a communication network such as the Internet or any other appropriate communication network. A client computer may make requests to server computers including requests for data. As an example, a client computer can request a video stream from a server computer associated with a movie streaming service. As another example, a client computer may request data from a database server. A client computer may comprise one or more computational apparatuses and may use a variety of computing structures, arrangements, and compilations for performing its functions, including requesting and receiving data or services from server computers.
[0017] A “memory” may refer to any suitable device or devices that may store electronic data. A suitable memory' may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method.Examples of memories including one or more memory' chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and / or magnetic mode of operation.
[0018] A “processor” may refer to any suitable data computation device or devices. A processor may comprise one or more microprocessors working together to achieve a desired function. The processor may include a CPU that comprises at least one high-speed data processor adequate to execute program components for executing user and / or system generated requests. The CPU may be a microprocessor such as AMD’s Athlon, Duron and / or Opteron; IBM and / or Motorola’s PowerPC; IBM’s and Sony’s Cell processor; Intel’s Celeron, Itanium. Pentium. Xenon, and / or Xscale; and / or the like processor(s).5KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0019] A “feature” can be an individual measurable property7or characteristic of a phenomenon. One or more features can be described using a “feature vector,” e.g., a structured list of data (such as numerical data) representing those features. A feature can be input into a model to determine an output. As an example, in pattern recognition and machine learning, a feature vector can comprise an / 7-dimensional vector of numerical features that represent some object. In some machine learning contexts, a numerical representation of objects can facilitate processing and statistical analysis. For image processing, for example, feature values might correspond to the pixels of an image. As another example, when feature vectors represent text, the features may comprise occurrence frequency of textual terms, or embeddings generated based on textual terms. Feature vectors can be equivalent to the vectors of explanatory variables used in statistical procedures such as linear regression.
[0020] A “dataset” may include any set of one or more “observations” or “data values.” A “data value” can include any data element. A “data element” can refer to a set of data that can be grouped into a single unit, enabling comparison between that data element and other data elements. For example, a data element can comprise a single numerical value (e.g., the speed of a vehicle in miles per hour) or could comprise multiple numerical values (e.g., 60 speed recordings of a vehicle corresponding to each minute of an hour-long period). As another example, a data element could comprise a digital document, or e.g., a “chunk” of a digital document. Data elements comprising multiple data values can be organized into various forms or structures, including data vectors, data tables, and “data sequences” comprising e.g., ordered lists of data elements. A data element may comprise the input to a machine learning model, and individual data values within that data element may comprise features. A data value can comprise a “data vector,” one or more values (represented in vector form) corresponding to a data element or observation.
[0021] “Sampling” may include any process or method used to collect data values.Sampling can be used to collect data values from an existing dataset. The act of sampling may result in a “sample,” one or more data values collected from the dataset during sampling. Datasets can be sampled via a variety7of means. For example, “random sampling” involves sampling data values from a dataset randomly. A “window” or “window of data” may include any number of contiguous data elements from a dataset. A “window” may be defined6KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 by a starting data value and an ending data value, such that the window contains all data values between the starting data value and ending data value (and optionally the starting data value and ending data values themselves). “Window sampling” can be used to sample data values contained within a window of data. A “stride” may refer to the rate at which a window “moves” across a sequence of data during e.g., “rolling window sampling” of that sequence of data. For example, with a stride of one, a window may move one data element “forward” in a sequence of data during each sampling operation, while with a stride of three, a window may move three data elements “forward” in a sequence of data during each sampling operation.
[0022] The term “artificial intelligence model” or “machine learning model” can include a model that may be used to predict outcomes to achieve a pre-defined goal. A machine learning model may be developed using a learning process, in which training data is classified based on known or inferred patterns.
[0023] " Machine learning" can include an artificial intelligence process in which software applications may be trained to make accurate predictions or otherwise generate outputs through learning. The predictions or outputs can be generated by applying input data to a model formed by performing statistical analyses on aggregated data. A model can be trained using training data, such that the model may be used to make accurate predictions or generate other outputs (e.g., images, natural language responses to input prompts, etc.) that are consistent with expectations. The prediction can be, for example, a classification of an image (e.g., identifying images of cats on the Internet), or as another example, a recommendation (e.g., a movie that a user may like or a restaurant that a consumer might enjoy).
[0024] A “machine learning model” (ML model) can refer to a software module configured to be run on one or more processors to provide a classification or numerical value of a property' of one or more samples or generate some other output. An ML model can include various parameters (e.g., for coefficients, weights, thresholds, functional properties of function, such as activation functions). As examples, an ML model can include at least 10, 100, 1,000, 5,000, 10,000, 50,000, 100,000, or one million parameters. An ML model can be generated using sample data (e.g., training samples) to make predictions on test data or otherwise generate outputs corresponding to test data. Various number of training samples can be used, e.g.. at least 10, 100, 1,000. 5,000, 10,000, 50,000. 100,000, or at least 200,0007KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 training samples. One example is an unsupervised learning model. Examples of unsupervised learning models include hidden Markov model (HMM), clustering (e.g., hierarchical clustering, k-means. mixture models, model-based clustering, density-based spatial clustering of applications with noise (DBSCAN), and OPTICS algorithm), approaches for learning latent variable models such as Expectation-maximization algorithm (EM), method of moments, and blind signal separation techniques (e.g., principal component analysis, independent component analysis, non-negative matrix factorization, singular value decomposition), and anomaly detection (e.g., local outlier factor and isolation forest).Another example type of model is supervised learning. Example supervised learning models may include different approaches and algorithms including analytical learning, statistical models, artificial neural network (e.g. including convolutional and / or transformer layers) that may have 1-10 layers as examples, recurrent neural network (e.g., long short term memory, LSTM), boosting (meta-algorithm), bootstrap aggregating (bagging) such as random forests, support vector machine (SVM), support vector (SVR), Bayesian statistics, case-based reasoning, decision tree learning, inductive logic programming, linear regression, logistic regression, Gaussian process regression, genetic programming, group method of data handling, kernel estimators, learning automata, learning classifier systems, minimum message length (decision trees, decision graphs, etc.), multilinear subspace learning, naive Bayes classifier, maximum entropy classifier, conditional random field, nearest neighbor algorithm, probably approximately correct learning (PAC) learning, ripple down rules, a knowledge acquisition methodology, symbolic machine learning algorithms, subsymbolic machine learning algorithms, minimum complexity machines (MCM), ordinal classification, data pre-processing, handling imbalanced datasets, statistical relational learning, or Proaftn (a multicriteria classification algorithm), or an ensemble of any of these types. Supervised learning models can be trained in various ways using various cost / loss functions that define the error from the known label (e.g., least squares and absolute difference from known classification) and various optimization techniques, e.g., using backpropagation, steepest descent, conjugate gradient, and Newton and quasi-New ton techniques.
[0025] The process of "‘training” a machine learning model may include any steps used to prepare a machine learning model to perform some task. Often training involves determining or optimizing a set of “parameters” (which characterize the machine learning model) that result in acceptable model performance. Training can be performed in a series of “training8KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 rounds” during which training data is used to update the parameters of the machine learning model, for example, based on a loss value.
[0026] A “loss”, “loss value”, or “error value” may include any value that indicates the deviation between a result of some process, method, or function and an expected, desired, or correct result. For example, if a machine learning model can detect anomalies in a dataset comprising 100 data values, 17 of which are anomalous, if the machine learning model only detects 15 of the 17 anomalous data values, the loss value could comprise, e.g., 2 (17 - 15). Loss values can be used to train and evaluate the training of machine learning models, e.g., by optimizing machine learning model parameters by minimizing the loss value, using processes such as stochastic gradient descent or backpropagation.
[0027] A “hy perparameter” can include any value used to configure a machine learning model that is external to the machine learning model. Typically, a hyperparameter is set and is not estimated or determined from the training data that is used to tram the machine learning model.
[0028] A machine learning model may comprise multiple “sub-models”, “layers,” or “modules”, which may refer to parts of a larger machine learning system. For example, a machine learning model could comprise a long short-term memory layer (which itself can comprise multiple layers), in addition to an attention layer and a linear layer. Layers can sometimes be organized in series, such that the input to a machine learning system is processed by a first set of layers, which produces an output that is then processed by a subsequent set of layers, and so forth until the output of the machine learning model is produced by the final layer in the series.
[0029] An “embedding” can refer to a representation of data, usually within an “embedding space”, a theoretical region in which embeddings can be compared via vector operations. For example, an embedding can comprise a vector representation of an image, which can be used to evaluate the similarity of that image to other images, or e g., determine whether that image contains or depicts a particular subject (e.g., a cat). Embeddings can be used within the field of machine learning, thereby enabling machine learning models to perform certain tasks, particularly tasks that are difficult or subjective. In some cases, an embedding can comprise a lower-dimensional representation of a corresponding element of data, enabling more9KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 efficient processing due to the reduced logical size of the embedding relative to the logical size of the original element of data.
[0030] An "‘encoder” can refer to something (e.g., a physical device, software element, component of a machine learning model, etc.) that produces output data representative of some input, usually in the form of a “code” or signal. An encoder can be used to convert data from one format to another, which may facilitate data processing. For example, a nominal data encoder can be used to convert nominal data (such as the name of a city) into a numeric code, facilitating such data to be processed using numerical methods. As another example, a physical rotary encoder can be used to convert the motion or position of a shaft or axle into electrical signals, facilitating computer-based odometry. A code (or other output of an encoder) can be used as an embedding. Some encoders can be implemented via applications of machine learning, e.g., via a “transformer” machine learning model or via another model that uses the principle of machine learning “attention.”BRIEF DESCRIPTION OF THE DRAWINGS
[0031] FIG. 1 shows a retrieval-augmented generation (RAG) framework according to some embodiments.
[0032] FIG. 2 shows various example prompts for pretraining, instruction-tuning and inference.
[0033] FIG. 3 shows a client-server system according to some embodiments.
[0034] FIG. 4 shows a compression learning process according to some embodiments.
[0035] FIG. 5 shows a flowchart of a training method according to some embodiments.
[0036] FIG. 6 shows a flowchart of an inference method according to some embodiments.
[0037] FIG. 7 shows pseudocode corresponding to a query expansion and novelty-based evidence selection method according to embodiments.
[0038] FIG. 8 shows histograms of the distributions of numbers of tokens for datasets used in experiments involving embodiments.
[0039] FIG. 9 shows histograms of the distributions of numbers of tokens according to Mistral-7B’s tokenizer for datasets used in experiments involving embodiments.10KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0040] FIG. 10 shows a table summarizing the performance of methods and systems according to embodiments (SARA) on various experimental datasets.
[0041] FIG. 11 shows a table comparing the performance of methods according to embodiments and baseline RAG methods on the SQuAD-v2 dataset.
[0042] FIG. 12 shows a table comparing the performance of methods according to embodiments and baseline RAG methods on the QASPER and QuALITY datasets with a context length budget of 512 tokens.
[0043] FIG. 13 shows a table comparing the performance of methods according to embodiments and baseline RAG methods on the NarrativeQA, SQuAD-v2. TriviaQA, and HotpotQA datasets with a context length budget of 512 tokens.
[0044] FIG. 14 shows a table comparing the performance of methods according to embodiments and baseline RAG methods on various experimental datasets with token budgets of 512 tokens and 1024 tokens.
[0045] FIG. 15 shows a comparison of query’ responses generated using different compression methods, including compression methods according to embodiments (SARA).
[0046] FIG. 16 shows graphs comparing the performance of methods according to embodiments and baseline RAG methods for five model architectures and across four LLM-based metrics on the QASPER dataset.
[0047] FIG. 17 shows graphs evaluating generalizability across models for embodiments and baseline RAG methods based on Fl and ROUGE-L scores for three experimental datasets.
[0048] FIG. 18 shows a table demonstrating the generalizability of methods according to embodiments to different retriever modules.
[0049] FIG. 19 shows graphs corresponding to an ablation study of embodiments, demonstrating the effect of different aspects of embodiments on Fl scores for three experimental datasets.11KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0050] FIG. 20 shows graphs demonstrating the sensitivity of methods according to embodiments to numbers of natural language context data elements for various experimental datasets.
[0051] FIG. 21 shows sample responses generated when using various numbers of natural language context data elements.
[0052] FIG. 22 provides a table summarizing compressor embedding models that can be used methods according to embodiments (and experiments involving methods according to embodiments) and their embedding sizes.
[0053] FIG. 23 shows graphs summarizing the effects of different compressor embedding models on Fl and ROUGE-L scores for various experimental datasets.
[0054] FIG. 24 shows a table summarizing evaluation results of embodiments (SARA) on out-of-domain datasets in terms of various LLM metrics.
[0055] FIG. 25 shows a table comparing textual context decoded from compression vectors and the original textual context used to generate those compression vectors.
[0056] FIG. 26 shows an additional table comparing textual context decoded from compression vectors and the original textual context used to generate those compression vectors.
[0057] FIG. 27 shows a graph showing how the number of words generated from compression vectors varies based on the number of sentences in reference textual context data elements.
[0058] FIG. 28 shows an example computer system according to some embodiments.DETAILED DESCRIPTION
[0059] As summarized above, embodiments of the present disclosure relate to a unified and efficient RAG framework, which is also referred to herein as "‘Selective and Adaptive Retrieval Augmented Generation with Context Compression”, or “SARA”. Embodiments are generally model agnostic, and do not require cumbersome architectural changes to underlying foundation model (e.g., LLM) backbones. Experiments across five LLM backbones (spanning three model families), retriever modules, and embedding models, demonstrate the12KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 effectiveness and generalizability of embodiments, e.g., that methods according to embodiments consistently improve answer relevance, answer correctness, and semantic similarity over existing RAG systems. Such experiments are described in the Experimental Evaluation section further below.
[0060] Embodiments of the present disclosure can enhance both retrieval and generation through structured evidence compression and adaptive data selection. This structured evidence compression can comprise part of a multi-level compression method that effectively balances local and global information in retrieved evidence, enhancing content relevance and coherence. In slightly more detail, locally precise context information (e.g., context information that is very relevant to a given prompt) can be provided to a foundation model (e.g., an LLM) in uncompressed form, while more globally relevant context information can be provided to the foundation model as "context compression vectors’" (e.g., compressed to the size of a single token in an LLM’s token space), greatly reducing token consumption. Adaptive data selection can be accomplished using iterative evidence selection methods that can be used to dynamically optimize retrieved information, e.g., by reducing redundancy in contextual information and improving retrieval of query -relevant contextual information.
[0061] Embodiments of the present disclosure and context compression vectors generated using methods according to embodiments can be used for various tasks beyond generating requestor query responses using LLMs. For example, embodiments of the present disclosure can be used in tasks such as data element re-ranking and citation resolution. Embodiments can also be applied to specialized domains (e.g., legal or scientific retrieval), where concise and faithful evidence representation can be important.
[0062] Prior to describing embodiments of the present disclosure in more detail, it may be helpful to describe RAG and prior attempts to address problems with RAG, including the use of context compression.
[0063] As summarized above, retrieval-augmented generation (RAG) can be used to aid in knowledge-intensive LLM tasks. Instead of treating LLMs as knowledge repositories, RAG methods involve generating responses to prompts using an external knowledge base (Lewis et al., 2020; Sharma et al., 2024), e.g., by retrieving relevant contextual information from the external knowledge database, then providing that contextual information to LLMs. A RAG system can comprise a “retriever” component 7?. which can be used to retrieve relevant data13KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 from a corpus C (e.g., stored in a database) e.g., based on a query q (e.g., provided by a user), e.g., the top n most relevant context data elements ct(e.g., documents, sections of documents, etc.) from the corpus C, i.e., {cit..., cn] c C. A "generator'' component (e.g., comprising an LLM or other foundation model) can synthesize any retrieved data in order to answer the query.
[0064] By retrieving relevant contextual information, RAG systems address model knowledge cutoffs and insufficient training coverage. Unfortunately, despite its promises, RAG still faces various challenges related to effectively retrieving, selecting, and integrating external evidence. As an example, LLMs are generally limited in the number of tokens they can receive as inputs (i.e., their context windows) due to the quadratic scaling of underlying transformer models. As a result, in some cases it may not be possible to provide sufficient relevant contextual information to LLMs using RAG. Further, even LLMs with extended context windows can struggle to process long, chunked retrieve contextual data effectively (Yu et al., 2024). The effective context of LLMs may be further limited (Yu et al. 2024) due to uneven attention distribution, as LLMs are often biased toward earlier tokens (Li et al. 2024b). making them sensitive to input order and prone to overlooking information near the end of input prompts (Yu et al., 2024). Further, expanding usable context in LLMs can require costly, model-specific architectural changes (Ding et al., 2023).
[0065] As another example, data retrieved using RAG systems often have redundant or loosely structured content (Yu et al., 2024; Ge et al.. 2024), particularly in documents such as transcripts or news articles. Such duplicate or irrelevant contents increases token usage, distracts generative Al models, degrades answer quality, and can lead to model hallucinations and a reduction in LLM accuracy.
[0066] As another example, existing context compression techniques for RAG can reduce input length (and therefore the number of tokens), but often also lead to a loss of fine-grained detail (e.g., for numeric values, organization names, and geographical locations). This can lead to hallucinated or incomplete responses by LLMs that use such data to generate their responses. Thus, while existing methods (including some of those described below) can achieve high compression rates, such methods risk discarding information needed for factual accuracy.14KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0067] Recent works such as Raptor (Sarthi et al., 2024), GraphRAG (Edge et al., 2024) and GraphReader (Li et al.. 2024a) focus on improving the retrieval and augmentation stages of RAG by structuring retrieve content and enhancing RAG through semantic or graph-based organization of knowledge, leading to more relevant and compact inputs for generation. Other approaches involve the use of ‘‘context compression”, which can refer to various techniques and methods for compressing contextual data used in RAG and can be used to reduce inference costs and maintain language understanding capabilities in long-context scenarios (Pan et al., 2024) or multi -turn scenarios (Kim et al., 2024a). Some existing RAG methods approach this using natural-language (NL)-based compression and / or representationlevel compression. NL-based compression methods (Zhang et al., 2024b; Chirkova et al. 2025) like ADACOMP (Zhang et al., 2024a), COMPACT (Yoon et al., 2024), and EXIT (Hwang et al. 2024) condense prompts or histories into concise natural language summaries, typically using extractive or abstractive summarization. These methods are generally modelagnostic and applicable across both open-source and proprietary LLMs (Zhu et al., 2025). Representation-based methods (Chevalier et al., 2023; Munkhdalai et al., 2024; Louis et al., 2025b, a), on the other hand, treat the LLM as a white box and modify attention calculations (Munkhdalai et al., 2024), positional encodings (Jin et al., 2024, Zhang et al., 2024c), or embeddings (Cheng et al., 2024). Methods such as xRAG (Cheng et al., 2024), GIST (Mu et al., 2024), and ICAE (Ge et al., 2024) project instruction demonstrations or contextual data into the language models' space.
[0068] By contrast, embodiments improve both retrieval and generation through structured evidence compression and adaptive selection, balancing local precision and global knowledge coverage under various context budgets. Some embodiments combine natural language text snippets with semantic “context compression vectors” to enhance context efficiency and answer correctness. These natural language text snippets can preserve fine-grained details that are typically lost by existing context compression techniques, while context compression vectors according to embodiments can summarize high-level semantics in a compact and interpretable manner.
[0069] More specifically, from the generation perspective, methods and systems according to embodiments can represent long contextual data using a small number of semantically rich, self-contained context compression vectors, which can act as lightweight abstractive15KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 summaries that preserve essential information while significantly reducing input length. Embodiments can leverage state-of-the-art embedding models (Meng et al.. 2024;Muennighoff et al., 2023) to encode retrieved context data into context compression vectors. These context compression vectors are explainable and can be interpreted through autoencoding to reveal their underlying semantics.
[0070] From the retrieval perspective, embodiments introduce an iterative evidence selection mechanism that can leverage generated compression vectors to dynamically refine a set of top-ranked context data elements. In methods according to embodiments, a computer system can progressively select contexts based on knowledge required to properly address a query and knowledge coverage of existing contexts, reducing redundancy while maximizing informativeness of query relevant content.
[0071] These and other aspects of embodiments are described in the following section, which describes methods and systems according to embodiments in more detail. It should be understood that any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform various steps described herein. Thus, some embodiments are directed to computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing a respective step or a respective group of steps.
[0072] Afterwards, experiments involving embodiments of the present disclosure are described. These experiments demonstrate the advantages of embodiments over the various prior RAG methods described above. Additionally, a description of an example computer system according to embodiments is described with reference to FIG. 28 and a list of references are provided.I. METHODS AND SYSTEMS
[0073] A RAG framework and an inference method according to embodiments (e.g., involving the generation of a query response for a query) are described below with reference to FIG. 1. Additionally, as embodiments may be particularly applicable to machine learning as a service (MLaaS) applications, a client-server system is described below with reference to FIG. 3, which involves a server computer according to embodiments generating query responses to requestor queries generated by requestor client computers. Afterwards, training16KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 methods according to embodiments are described in more detail with reference to FIG. 4 and the flowchart of FIG. 5. Additionally, inference methods according to embodiments (including methods for query expansion and novelty-based evidence selection) are described in more detail with reference to the flowchart of FIG. 6 and the pseudocode of FIG. 7.A. RAG Framework
[0074] FIG. 1 shows an RAG framework according to embodiments. During inference, the RAG framework can be used to generate a query response 102 to a query 104 using a foundation model 106 (e.g., an LLM or another generative Al model). In cases where the foundation model 106 comprises an LLM, the query 104 can comprise a natural language question or statement, such as '’how large is the ANTI-SCAM dataset?”, e.g., represented as data corresponding to a text or character encoding standard. Similarly, the query response 102 can comprise a natural language response to the question or statement contained in query 104. In some cases, the query 104 may be posed by a “requestor”, such as a user, or a client computer operated by the user. As such, the query 104 may sometimes be referred to as a “requestor query” and the query response 102 may sometimes be referred to as a “requestor query response”.
[0075] The RAG framework may be instantiated and / or used by a computer system according to embodiments, e.g., a computer system may execute software code, instructions, etc., in order to perform the functions associated with the various elements of the RAG framework depicted in FIG. 1, e.g., using a retriever software module or software library to perform functions associated with retriever module 108.
[0076] In more detail, the retriever module 108 (or “retriever”) can be used retrieve relevant context data elements 110 from a database 112 (or “corpus”) based on the query 104, e.g., based on keyword searching, vector-based comparison (e.g., in which a vector representation of the query is compared against vector representations of context data elements in the database 112), or any other appropriate technique. Such context data elements 110 could comprise, e.g., documents or other datafiles from the database 112, or “chunks” of documents or data files, e.g., a relevant section, page, paragraph, sentence, etc., from a document. In some embodiments, context data elements 110 can comprise the "lop- / ?” most relevant context data elements from the database 112, e.g.. the top 20 most relevant17KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 context data elements according to the retriever module 108, e.g., based on a keyword search, vector-based comparison, etc. performed by the retriever module 108.
[0077] A computer system can compress the context data elements 110 using a compression model 114 in order to generate context compression vectors 116. As single context compression vectors may have limited representational capacity, in some embodiments, each retrieved context data element can comprise a chunk, and each context compression vector can encode a single chunk of data. The compression model 114 can comprise an embedding layer 118 and a projection layer 120. In general, embedding layer 118 can be used to generate embeddings from context data elements 110, e.g., by converting chunks of text (e.g., individual sentences) into embeddings using sentence embedding methods. Embodiments of the present disclosure can use various state-of-the-art embedding models (e.g., from Meng et al.. 2024; Muennighoff et al., 2023) to implement embedding layer 118 and generate embeddings. In general, the projection layer 120 can be used to project the embeddings generated using the embedding layer 118 to the token space of the foundation model 106, thereby generating context compression vectors 116. Such context compression vectors 116 may have a smaller data size than either their respective context data elements (or respective chunks of those context data elements) or even the vector embeddings generated using embedding layer 118.
[0078] At step 128, the context data elements 110 and context compression vectors 116 may be used to generate a prompt, which can subsequently be evaluated using foundation model 106 to generate query response 102. However, prior to this step, a computer system according to embodiments may perform context selection and re-ranking methods according to embodiments at step 122. In general terms, step 122 can be thought of as a “fine-grained” retrieval phase, complementing “coarse” retrieval performed using retriever module 108. This contrasts with previous RAG methods, which often use 'rein eve-arid- read’ pipelines, which often incorporate redundant context data and overlook interdependencies between previously retrieved context data and newly needed information (Wang et al., 2024). In long-context understanding, what should be retrieved next is often dependent on what has already been inferred from previously retrieved evidence (Sarthi et al., 2024; Li et al., 2024a).
[0079] More specifically, in step 122, from among the n context data elements 110 retrieved by the retriever module 108, a computer system according to embodiments can18KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 identify the k most relevant context data element, e.g., the top five context data elements from among 20 context data elements 110, which may be referred to as “primary context data elements’7130. These most relevant context data elements can be identified via embeddingbased novelty metrics 124 and conditional self-information metrics 126. Such metrics are described in more detail further below. In general however, embedding-based novelty metrics 124 can be used to evaluate which context data elements are relevant by comparing their context compression vectors against a vector representation of the query 104, e.g.. by determining which set of context compression vectors minimizes the difference (e.g., the Euclidean distance) between the query7vector and that set of context compression vectors. By contrast, conditional self-information metrics can be used to evaluate which set of context data elements provides the “most information” based on the surprisal of context data elements.
[0080] V ia step 122, the computer system can identify one or more primary context data elements 130 and one or more secondary context data elements 132, e.g., where the primary context data elements 130 comprise the top k context data elements from among the n context data elements 110 retrieved by retriever module 108, and the secondary context data elements 132 comprise the remaining n-k context data elements, e.g., determined based on the embedding-based novelty metrics 124 and / or the conditional self-information metrics 126.
[0081] As indicated in FIG. 1, the primary context data elements 130 and secondary context data elements 132 can be used to construct a prompt for the foundation model 106 in step 128. Notably, the primary context data elements 130 can be included in the prompt in textual form, e.g., as uncompressed text, while the secondary context data elements 132 can be included in the prompt as context compression vectors 134 (which can comprise tokens in the foundation model’s 106 token space). In this way, embodiments can provide relevant, locally precise context information (e.g., the primary context data elements 130) in uncompressed form, while providing more globally relevant context information (e.g., the secondary context data elements 132) in compressed form, efficiently utilizing the foundation model’s 106 limited context window with the most relevant information.
[0082] The prompt generated in step 128 can also include various other data, including a textual preamble 136 (e.g., a set of instructions for foundation model 106) and the query 104. FIG. 2 shows another example of a prompt template 202 that can be used during inference.19KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001The prompt can then be evaluated using the foundation model 106 (which can be provided by an external model provider in some embodiments), thereby generating the query response 102. If applicable, the query response 102 can be returned to a requestor associated with query 104, e.g., a client computer or a user of a client computer. During training (or during experiments to evaluate the effectiveness of the RAG framework), the query response 102 can be used to derive a loss 138, which can be used to train the various trainable components of the RAG framework (e.g.. the foundation model 106 and the projection layer 120), as described in more detail further below.
[0083] A notable advantage of the RAG framework of FIG. 1 is that it is largely component-agnostic, e.g., the various components (e.g., foundation model 106, retriever module 108, compression model 114, embedding layer 118, projection layer 120, etc.) can be replaced without affecting the performance of the system (assuming that the replacements are sufficiently functional). This is an advantage because the RAG framework of FIG. 1 can incorporate newer, more effective foundation models 106, retriever modules 108, compression models 114, etc., as they are developed. However, it may be advantageous to use the same components for both training (e.g., compression learning and instruction-tuning, as described further below) and inference, as context compression vectors 116 corresponding to a given foundation model’s 106 token space may not be applicable to a different foundation model’s token space.B. Client-Server System
[0084] As mentioned above, embodiments of the present disclosure may be useful in machine learning as a service (MLaaS) systems, in which a computer system according to embodiments generates requestor query responses to requestor queries on behalf of requestors. FIG. 3 shows an exemplary client-server system, including a computer system 302 (e.g., a server computer) that can perform methods according to embodiments for client computer(s) 310 (which can comprise requestor(s)). As described in more detail below with reference to FIG. 28, computer system 302 can comprise one or more processors (not pictured) and a non-transitory computer readable medium (e.g., a hard drive) coupled to the one or more processors (also not pictured). The non-transitory computer readable medium can comprise code or instructions, executable by the processor for performing methods according to embodiments described herein.20KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0085] Client computer(s) 310 and computer system 302 can communicate over a communication network 312. A communication network such as communication network 312 can take any suitable form, and may include any one and / or the combination of the following: a direct interconnection; the Internet; a Local Area Network (LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodes on the Internet (OMNI); a secured custom connection; a Wide Area Network (WAN); a wireless network (e.g., employing protocols such as. but not limited to a Wireless Application Protocol (WAP), I-mode, and / or the like); and / or the like. Messages between computers and devices in the system of FIG. 3 and / or over communication network 312 may be transmitted using a secure communication protocol, such as, but not limited to, File Transfer Protocol (FTP); HyperText Transfer Protocol (HTTP); Secure HyperText Transfer Protocol (HTTPS); Secure Socket Layer (SSL), ISO (e.g., ISO 8583) and / or the like. Any suitable communication protocol can be used to communicate over the communication network 312, e.g., for the purpose of creating one or more communication channels. A communication channel may, in some instances, comprise a secure communication channel, which may be established in any known manner, such as through the use of mutual authentication, a session key, and establishment of a Secure Socket Layer (SSL) session.
[0086] In such communications over communication network 312, a client computer 310 may transmit a requestor query to computer system 302. Such a requestor query could comprise a natural language query such as “what is the size of the ANTI-SCAM dataset?”. As described above with reference to FIG. 1, computer system 302 can use a retriever module 316 and the requestor query in order to retrieve context data elements from a database 308 (or other corpus or data source, e.g., the Internet). Computer system 302 can then use a compression model 318 to generate context compression vectors based on the retrieved context data elements. Computer system 302 can then perform context selection and reranking methods according to embodiments using context selection and reranking module 320. Using prompt generation module 322, computer system 302 can generate a prompt comprising primary context data elements (e.g., in uncompressed textual form) and secondary context data elements (e.g., as context compression vectors). Computer system 302 can then generate a requestor query response using the prompt and the foundation model 304.Computer system 302 can return the requestor query response to client computer 310 over communication network 312. Alternatively, in some embodiments, foundation model 30421KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 may be provided by an external model provider server 324 (e.g., an LLM provider). In such embodiments, computer system 302 can transmit the prompt to external model provider server 324 (e.g., via communication network 312). External model provider server 324 can then use foundation model 304 to generate the requestor query response, which can be provided back to computer system 302 (and then provided to client computer 310), or which can be directly provided to client computer 310.
[0087] It should be appreciated that the numbers of devices, entities, components, models, modules, etc., shown in FIG. 3 were selected for ease of illustration and explanation. It should be understood that systems according to embodiments of the present disclosure can include more than one of each device, entity, component, computer system, etc. In addition, some systems according to embodiments may include a lesser number of devices, entities, and / or components or a greater number of devices, entities, and / or components than those shown in FIG. 3. For example, computer system 302 may comprise a distributed computing system comprising several computers collectively performing methods according to embodiments. Likewise, there may be multiple databases 308 from which computer system 302 retrieves context data elements, along with multiple communication networks 312 over which computer system 302 communicates with client computers(s) 310. It should be understood that the particular modules depicted in FIG. 3 (and their configuration) are intended to represent only one possible implementation, selected for ease of explanation, and are not intended to be limiting. Various other configurations may become apparent upon reading this disclosure, e.g., implementing methods according to embodiments using a single monolithic software module.
[0088] Having described a RAG framework according to embodiments and a client-server system according to embodiments, it may be helpful to describe training and inference methods according to embodiments in more detail below.C. Training
[0089] Some embodiments of the present disclosure may make use of a pre-trained foundation model to generate query responses to queries, e.g., an LLM provided by an external model provider or a pre-trained model with open-source weights. As such, while in some embodiments a foundation model can be trained, it may not be necessary in other22KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 embodiments. However, other components of RAG frameworks and systems according to embodiments may be trained for various purposes. For example, a compression model (e.g., compression model 114 from FIG. 1) may be trained to generate context compression vectors that effectively map context data elements to a token space of a foundation model. This may be accomplished using “compression learning'’ as described below.
[0090] Additionally, in some embodiments, a foundation model can comprise a pre-trained foundation model that can be instruction-tuned to generate query responses to prompts using a low-rank adaptation module (LoRA) (Hu et al., 2021). This can adapt the foundation model to improve the foundation model’s performance on prompts comprising both textual context (e.g., corresponding to primary context data elements) and compressed context (e.g., comprising compression vectors corresponding to secondary context data elements). In some embodiments, foundation models can be instruction-tuned using pre-computed context data elements retrieved via BM25 (Robertson et al., 2004).
[0091] Training methods according to embodiments are largely model-agnostic, and are compatible with various retriever modules, embedding models, and foundation models (with and without parameter access). By using a compression model external to a foundation model, the context embedding space can be aligned with the foundation model’s token space, without requiring changes to internal components like the attention mechanism (in transformer-based foundation models). This enables the use of more powerful embedding models and foundation models as they are developed in the future.1. Compression Learning
[0092] During compression learning, a computer system according to embodiments can train a compression model to generate training context compression vectors from training context data elements (and sequences of training context data elements, which may be referred to as “training context data sequences”) that can be used to accurately reconstruct those training context data elements (or training context data sequences). As a result of this training, context compression vectors generated by compression models according to embodiments are interpretable by foundation models during inference, provide semantic fidelity by preserving sufficient information for accurate context reconstruction, and require minimal adaptation across retriever modules and foundation models. Unlike traditional23KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 extractive or abstractive summarization methods (e.g., Xu et al., 2024) that require multiple passes, these context compression vectors naturally serve as high-ratio, parallelizable summaries. FIG. 4 shows a diagram related to a compression learning process 402 according to some embodiments. In some embodiments, the compression learning process 402 can comprise an auto-encoding task (Liu et al., 2023; Cheng et al., 2024).
[0093] In some embodiments, the compression learning process 402 can be further divided into embedding alignment learning 404 and context reconstruction learning 406. In summary, embedding alignment learning 404 can involve training the compression model 408 to generate training context compression vectors 410 that can be interpreted by the foundation model 412 to contain the same meaning as a single chunk (e.g., a single page, paragraph, sentence, etc.) from a training context data element (e.g., from among training context data elements 420). Performance can be evaluated based on an alignment loss 422, which a computer system can determine by comparing output training context data elements 424 against training context data elements 420 used to generate the training context compression vectors 410. This alignment loss 422 can then be used to train the compression model 408 using stochastic gradient descent, backpropagation, or any other applicable lossbased training methods.
[0094] Context reconstruction learning 406 may be similar to embedding alignment learning 404, except rather than training compression model 408 to generate training context compression vectors 410 that can be used to reconstruct individual chunks from (or comprising) training context data elements 420, a computer system can train compression model 408 to generate training context compression vectors vector sequences 432 that can be used to reconstruct training context data sequences 430, e.g., comprising sequences of training context data elements from training context data elements 420, e.g., comprising entire digital documents. A computer system can use foundation model 412 to generate output training context data sequences 426 corresponding to training context compression vector sequences 432. These output training context data sequences 426 can be compared against training context data sequences 430 in order to produce a reconstruction loss 428, which can then be used to train the compression model 408. In some embodiments, the compression model can be trained to perform full context reconstruction in context reconstruction learning after performing embedding alignment learning 404.24KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0095] Training context compression vectors (e.g., training context compression vector 418) and training context compression vector sequences (e.g., training context compression vector sequence 426) can be encoded or otherwise included into training prompts that can be provided to the foundation model 412 during compression learning 402. FIG. 2 shows an example of a training prompt template 204 that can be used in embedding alignment learning 404, as well as a training prompt template 206 that can be used in context reconstruction learning 406.
[0096] Methods for training a compression model to generate context compression vectors in a training process (e.g., compression learning, as described above) may be better understood with reference to the flowchart of FIG. 5.
[0097] At step 502, a computer system can generate a plurality of training context compression vectors corresponding to a plurality of training context data elements using the compression model. This can be accomplished by first embedding the plurality of training context data elements using an embedding layer (which can implement various compressor embedding models, e.g., those listed in the table of FIG. 22) to generate embeddings (e.g., compact representations of individual chunks), then inputting those embeddings into a projection layer (e.g., comprising a multilayer perceptron) to generate the plurality of training context compression vectors. The plurality of training context compression vectors can correspond to a token space of a foundation model, e.g., each training context compression vector can comprise (or can be mapped to) a token.
[0098] The computer system can retrieve the plurality of training context data elements from any appropriate source, e.g., a training database (or corpus), the Internet, etc. Notably, as this training relates to aligning the embedding spaces of the compression model and the LLM, it is not necessary that the training context data elements match the domain of context data elements used during inference. As such, the plurality of training context data elements can be collected from any (reasonably high quality) natural language dataset. In some implementations of embodiments (and related experiments), a Wikipedia dataset (Izacard et al., 2023) was used, which offers extensive topical coverage and diverse narrative styles, and which has been shown to enhance models’ language understanding (Cheng et al., 2024; Gao et al., 2023).25KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0099] At step 504, the computer system can decode the plurality of training context compression vectors using a foundation model, thereby generating a plurality of output training context data elements. As an example, the computer system can decode the plurality of training context compression vectors by encoding the plurality of training context compression vectors into a plurality of training prompts (e.g., similar to training prompt template 204 from FIG. 2), then input those training prompts into the foundation model to produce a plurality of training responses, which can either comprise the plurality of output training context data elements or can be used to derive the plurality of output training context data elements.
[0100] At step 506, the computer system can determine an embedding alignment loss £alignbased on a difference between the plurality of training context data elements and the plurality of output training context data elements. As an example, the embedding alignment loss £aiign can be formulated as £align(s() = — log Pg(s(| Enc(s(), xins). where stis a context data element, Pgis the compression model or component(s) of the compression model (e.g., a linear projection layer and / or an embedding layer) with parameters 9, and xinsis a decoding instruction such as ‘‘The token < C> means: [CHUNK]” (e.g., as depicted in training prompt template 204 of FIG. 2).
[0101] At step 508, the computer system can generate a plurality of training context data sequences. Each training context data sequence can comprise one or more training context data elements, e.g., from the plurality of training context data elements. In cases in which training context data elements correspond to chunks of digital documents (or other contextual data), each training context data sequence can comprise those digital documents themselves, i.e., the entire contexts represented by the sequences of their respective training context data elements.
[0102] At step 510, the computer system can generate a plurality of training context compression vector sequences corresponding to the plurality of training context data sequences. This can be accomplished in a maimer similar to step 502. For example, the computer system can sequentially embed each training context compression vector in each training context compression vector sequence using an embedding layer to generate sequences of embeddings, then sequentially input those embeddings into a projection layer to generate training context compression vectors, which can be organized into the plurality of26KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 training context compression vector sequences. Such training context compression vector sequences can correspond to a token space of the foundation model, e.g., each training context compression vector in each training context compression vector sequence can comprise (or can be mapped to) a token.
[0103] At step 512, the computer system can decode the plurality of training context compression vector sequences using the foundation model, thereby generating a plurality of output training context data sequences. As an example, the computer system can decode the plurality of training context compression vector sequences by encoding the plurality of training context compression vector sequences into a plurality of training prompts (e.g., similar to training prompt template 206 from FIG. 2), then input those training prompts into the foundation model to produce a plurality of training responses, which can either comprise the plurality of output training context data sequences or can be used to derive the plurality of output training context data sequences.
[0104] At step 514, the computer system can determine a context reconstruction loss -^recons based on a difference between the plurality of training context data sequences and the plurality of output training context data sequences. As an example, the context reconstruction loss £reCons can be formulated as £reCOnsW =— log Pg(Sj | {Enc(ty), Vs(e c], %ins), where c is a training context data sequence (e.g., a document) comprising a sequence of multiple training context data elements {s;}, each of which can be encoded as a separate training context compression vector, and %insis a decoding instruction such as "interpret the following tokens as a single document: < C> < O... < C>: < Paragraph>” (e.g., as depicted in training prompt template 206 of FIG. 2).
[0105] At step 516, the computer system can update parameters of the compression model based on the embedding alignment loss and the context reconstruction loss, thereby training the compression model. Such parameters can comprise, e.g., the parameters of the projection layer and / or the embedding layer. The computer system can use any appropriate machine learning techniques to update the parameters of the compression model, e.g., stochastic gradient descent, backpropagation, etc. The training process of FIG. 5 can be repeated any number of times until training has been completed, e.g., according to a set number of training rounds or epochs, until a convergence condition has been achieved, etc.27KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0106] The computer system can update the parameters of the compression model based on the embedding alignment loss and the context reconstruction loss simultaneously, e.g., by generating a combined loss term based on these losses and then updating the parameters of the compression model based on this loss term. Alternatively, the computer system can perform embedding alignment learning before performing context reconstruction learning, e.g., updating the parameters of the compression model based on the embedding alignment loss, then later (e.g., after several training rounds) update the parameters of the compression model based on the context reconstruction loss. Further, in some embodiments, a “curriculum learning strategy ” (Bengio et al., 2009; Wang et al., 2021) can be used to improve training stability. More specifically, in some embodiments, the compression model can be initially trained during embedding alignment learning using short context data element chunks (e.g., short sentences) with a gradual transition into more complex examples, including entire contextual documents during context reconstruction learning.D. Inference
[0107] A method for generating a query response corresponding to a query using a foundation model and a plurality of context data elements can be better understood with reference to FIG. 1, the flowchart of FIG. 6, and the pseudocode of FIG. 7. Such a method can be performed by a computer system.1. Receive Query
[0108] At step 602, a computer system can receive a query, which can comprise, e.g., a natural language question or statement. FIG. 1 shows a query 104 comprising the text “how large is the ANTI-SCAM Dataset?”. If the computer system is performing methods according to embodiments as a service for requestors (e.g., as part of an MLaaS system), the query can comprise a requestor query, and the computer system can receive the requestor query from a requestor, e.g., over a communication network (such as the Internet) as depicted in FIG. 3. In some embodiments, the requestor can comprise a client computer and the computer system can comprise a server computer. In some embodiments, the computer system can generate a query vector corresponding to the query, e.g., by encoding the query using a compression model. This query vector can be used to identify relevant context data elements (e.g., at step 604), as well as determine primary' and secondary’ context data28KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 elements (e.g., at step 608), as described in more detail further below. In some embodiments, the query vector can comprise a combination of both an encoding of the query’ and an encoding of a most-relevant context data element (e.g., identified at step 604). As such, in some embodiments the computer system can generate the query vector in a subsequent step, e.g., after identifying a plurality of context data elements in step 604.2. Identify Context Data Elements
[0109] At step 604, the computer system can identify a plurality of context data elements based on relevance to the query. In some embodiments, identifying the plurality of context data elements can comprise searching a database of context data elements using a retriever module. Such context data elements can comprise relevant data elements from a corpus, e.g., stored in a database. In some embodiments, the plurality of context data elements can comprise a plurality of digital documents and / or chunks of digital documents. In some embodiments, determining the plurality of context data elements using the retriever module can involve the computer system generating a requestor query vector corresponding to the requestor query. The computer system can then perform a context data element search using the retriever module by comparing the requestor query vector to data element vectors representative of a plurality of data elements. The plurality of context data elements can comprise a plurality of relevant context data elements from among the plurality of data elements according to the context data element search. The computer system can then perform a “chunking” process, if necessary, in order to split up retrieved context data elements into smaller context data elements.
[0110] The first few retrieved documents could be very relevant to the query based on such vector comparisons. However, the first few retrieved documents could also be very redundant, e.g., each comprising the same page of similar editions of a textbook on a subject, which could be largely identical. Providing redundant context data elements to a foundation model (e.g., an LLM) as part of RAG might not be any more effective than providing a single context data element, but may increase cost and processing time due to the increased number of tokens. Hence, the ordering of the top n most relevant data elements, as provided by retriever module, may not necessarily be the most relevant or useful ordering for a foundation model. Further, recent studies have found that LLMs have an imbalanced focus on earlier29KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 input tokens, partly due to causal attention mechanisms (Zhang et al., 2024b; Li et al., 2024b). As a result, information appearing later in provided prompts may be underutilized.[OHl] To address these problems, in some embodiments, a computer system can perform further context selection and re-ranking (e.g., as described above with reference to step 122 of FIG. 1) to create a more useful ordering of retrieved context data elements. This reranking method can balance relevance and novelty, enabling a computer system to identify and include context data elements that are relevant to a query, provide novel information (thereby reducing redundancy), and are ordered in a manner that improve the performance of a foundation model. In this way, methods according to embodiments improve a foundation model’s ability to generate more accurate and informative answers. Dynamic re-ranking methods are described in more detail further below with reference to step 608 and FIG. 7. Generally, step 604 can be considered a "‘coarse” retrieval step, which can efficiently identify the (generally) most relevant context data elements and eliminate irrelevant data elements, and step 608 can comprise a “fine-grained” retrieval step, selecting data elements based on their informativeness, relevance, diversity, etc.3. Generate Context Data Element Compression Vectors
[0112] At step 606, the computer system can generate a plurality of context compression vectors (also referred to as “context data element compression vectors”) corresponding to the plurality of context data elements. In some embodiments, the computer system can generate the plurality’ of context compression vectors using a compression model, which can comprise an embedding layer and a projection layer, as described above with reference to FIGs. 1 and 4. Various embedding models can be used to implement the embedding layer, e.g., those shown in the table of FIG. 22. In some embodiments, the plurality of context compression vectors can correspond to a token space of the foundation model, e.g., the foundation model is able to interpret each compression vector as a token. In some embodiments, as described above with reference to FIGs. 4 and 5, the computer system can train the compression model, e.g., prior to generating the plurality of context compression vectors at step 606.30KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W0014. Determine Primary Context Data Elements, Ordering, and Second Context Data Elements
[0113] At step 608, the computer system can determine one or more primary context data elements of the plurality of context data elements, a determined order of the one or more primary context data elements, and one or more secondary context data elements of the plurality of context data elements. In some embodiments, the determined order can be based on relevance of each of the one or more primary context data elements to the query. In some embodiments, these primary context data elements, secondary context data elements, and the determined order can be determined based on the plurality of context compression vectors. In general, the primary context data elements can comprise more relevant context data elements that can be provided to a foundation model in uncompressed form (e.g., as textual data elements), while the secondary context data elements can comprise less relevant (but still relevant) context data elements that can be provided to the foundation model in compressed form (e g., using their respective context compression vectors).
[0114] Due to the limited effective context windows of some foundation models (e.g., LLMs) sometimes performance can degrade when relevant information is presented later in lengthy inputs (Jian et al., 2024). The determined order described above may be useful, as it allows the computer system to place more relevant primary context data elements earlier in prompts provided to foundation models, e.g., as described further below with reference to step 610.
[0115] In some embodiments, there may be a predetermined number of context data elements, primary context data elements, and secondary context data elements. For example, a number of primary context data elements in the one or more primary context data elements may be equal to a predetermined primary’ context count, and likewise a number of secondary context data elements in the one or more secondary context data elements can be equal to a predetermined secondary' context count. As a non-limiting example, a total number of context data elements n could comprise 20, a predetermined primary' context count k could comprise five (i.e., there are five primary context data elements), and a predetermined secondary’ context count n-k could comprise 15 (i.e.. there are 15 secondary context data elements). The number of context data elements, primary context data elements, and secondary context data elements can be established in any appropriate manner, e.g., as a set31KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 hyperparameter established by practitioners of methods according to embodiments, determined via automatic means, etc.
[0116] In some embodiments, the computer system can use iterative context selection to dynamically select context data elements and add them to a primary context data element set based on their incremental semantic value. Thus, in some embodiments, the one or more primary context data elements can comprise a primary context data element set, and the computer system can determine the one or more primary’ context data elements by determining the primary context data element set. In some embodiments, the computer system can identify a first context data element from the plurality of context data elements based on a relation of the first context data element to the query’, then generate a primary context data element set initially comprising the first context data element. In some embodiments, the first context data element can comprise a most relevant context element determined by using the retriever module, e.g., according to a context data element search performed e.g., at step 604.
[0117] After initializing the primary’ context data element set, the computer system can then perform one or more iterative rounds. Each iterative round can comprise updating the primary context data element set to include a selected context data element from one or more remaining context data elements of the plurality of context data elements, thereby generating an updated primary’ context data element set. Each selected context data element can be selected based on a comparison of each remaining context data element of the one or more remaining context data elements to the primary context data element set. In such embodiments, the determined order can comprise the order in which each selected context data element was added to the primary’ context data element set. Further, the one or more secondary context data elements can comprise one or more unselected context data elements from the plurality of context data elements that are not in the primary context data element set.
[0118] Expressed in other terms, in each round, the computer system can determine how suitable each context data element is for the primary’ context data element set, then add the most suitable context data element to the primary’ context data element set. This process can be repeated, e.g., for some predetermined primary context count. The determined order can comprise the order in which the selected context data elements were added to the primary’32KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 context data element set, and the secondary context data elements can comprise the context data elements that were not added to the primary context data element set.
[0119] There are various ways that the computer system can determine which context data element to select and include in the primary context data element set. As described above with reference to FIG. 1, one approach is the use of embedding-based novelty7(e.g., using embedding-based novelty7metrics 124). Another approach is the use of conditional selfinformation (e.g.. using conditional self-information metrics 126). In general, embeddingbased novelty can involve evaluating how much new (i.e., novel) information is contributed by context data elements based on their embeddings (e.g., context compression vectors), such that context data elements that provide more new information are selected to include in the primary context data element set. By contrast, conditional self-information can involve evaluating how much novel information is contributed by context data elements based on the surprisal of those context data elements given existing context data elements in the primary context data element set. Embedding-based novelty7and conditional self-information are described in more detail further below. Methods to determine the one or more primary context data elements (including the one or more iterative rounds), as well as the use of embedding-based novelty and conditional self-information, can be better understood with reference to the pseudocode of FIG. 7.
[0120] In line 702, a computer system can use a retriever module to retrieve the plurality of context data elements. Line 702 generally corresponds to step 602 of FIG. 6. Afterwards, at line 704, the computer system can generate a query vector based on the query, e.g., by encoding the query using a compression model. Specifically in line 704, the computer system can generate the query vector by averaging an encoding or embedding of the query and an embedding or encoding of the first context data element (e.g., a most relevant context data element determined using the retriever module), e.g., by encoding these data using a compression model. At line 706, the computer system can initialize the primary context data element set containing the first context data element.
[0121] The for-loop of lines 708-716 can correspond to the iterative rounds described above. More specifically, at line 712, the computer system can determine a selected context data element using “evidence selection”, e g., using embedding-based novelty or conditional self-information, which are represented formulaically in equations (3) and (4) of FIG. 7. If33KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 necessary (e.g., in order to determine the selected context data element at line 712), the computer system can aggregate context compression vectors corresponding to the primary context data element set at line 710, e.g., as described in more detail below in relation to embedding-based novelty. At line 714, after determining a selected context data element, the computer system can add the selected context data element to the primary' context data element set.a) Embedding-Based Novelty’
[0122] As described above, in some embodiments, the computer system can use embedding-based novelty' in order to identify the one or more primary context data elements. In general, embedding-based novelty can involve evaluating how much new (i.e.. novel) information is contributed by context data elements based on their embeddings (e.g., context compression vectors), such that context data elements that provide more new information are selected to include in the primary' context data element set. In such embodiments, the comparison of each remaining context data element to the primary' context data element set can comprise an embedding-based comparison that comprises minimizing a difference between the query vector and a comparison vector corresponding to the updated primary context data element set. In some embodiments, the comparison vector can comprise a combination of context compression vectors corresponding to the one or more primary context data elements in the updated primary’ context data element set.
[0123] More specifically, updating the primary context data element set to include a selected context data element can involve generating a candidate primary context data element set for each remaining context data element, thereby generating one or more primary' context data element sets. Each candidate primary context data element set can comprise the primary context data element and a different candidate context data element. The computer system can then generate a candidate set comparison vector for each candidate primary context data element set, thereby' determining one or more candidate set comparison vectors. Each candidate set comparison vector can comprise a combination of context compression vectors corresponding to one or more context data elements in that candidate primary context data element set. The computer system can evaluate the one or more candidate primary’ context data element sets by comparing each candidate set comparison vector to the query vector. In some embodiments, similarity' between comparison vectors and the query’ vector34KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 can be determined based on a Euclidean distance between each comparison vector and the query vector. The updated primary’ context data element set can comprise a candidate primary context data element set for which a corresponding comparison vector is most similar to the query- vector, and the selected context data element can comprise the different candidate context data element included in that candidate primary' context data element set.
[0124] This process may be better understood with reference to the following embeddingbased novelty formula: SelectEvi(q, Vsel, V) = argmin Hv,? — Aggregate({v G VselU )||2, which shoyvs hoyv a candidate context data element vtG V\Vselfrom among the remaining context data elements V can be selected and added to the primary context data element set Vselbased on a query vector vq. The computer system can generate a candidate context data element set VselU v, by adding the candidate context data element to the primary context data element set. The computer system can then generate a candidate set comparison vector Aggregate({v G VselU {vj}) which can comprise a combination (or aggregation) of context compression vectors corresponding to the one or more context data elements in the candidate primary context data element set. The computer system can then determine the Euclidean distance || vq— Aggregate({v G VselU {Vi}})||2between the query vector and the candidate set comparison vector, selecting the candidate context data element that minimizes this Euclidean distance and adding that candidate context data element to the primary’ context data element set. As described above, in some embodiments, as queries are usually fairly succinct, the query vector can be supplemented by aggregating the embeddings of both the query q and the first context data element v, i.e., vq= Avg(Enc(q), Enc^)).b) Conditional Self-Information
[0125] As an alternative, in some embodiments, the computer system can use conditional self-information (Shannon, 1948) in order to identify the one or more primary context data elements, e.g., by quantifying the surprisal of remaining context data elements in view of the context data elements in the primary context data element set. In such embodiments, the comparison of each remaining context data element to the primary' context data element set comprises a conditional self-information comparison that comprises maximizing selfinformation of a remaining context data element given the primary context data element set.35KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001The selected context data element can comprise the remaining context data element with maximized self-information.
[0126] More specifically, updating the primary context data element set to include a selected context data element can involve the computer system determining, for each remaining context data element, a conditional self-information value, thereby determining one or more conditional self-information values. The computer system can then determine a greatest conditional self-information value, then identify the selected context data element based on the greatest conditional self-information value. The selected context data element can comprise a remaining context data element corresponding to the greatest conditional selfinformation value. The computer system can add the selected context data element to the primary context data element set, thereby generating the updated primary context data element set.
[0127] This process may be better understood with reference to the following conditional self-information formulas SelectEvi(<7, Vsel, V = argmax / (v; |Vsel) and I(fy |Vsel) =tyEV\Vsei— logP(iv |vj e Vsel, ivf, w-1), where / (v; |Vsel) is the conditional self¬information of a remaining context data element vLgiven the primary context data element set Vsel. In some embodiments, the conditional self-information value for each remaining context data element can be estimated with a small proxy language model for efficiency. Remaining context data elements with higher conditional self-information values introduce more novel information, while lower conditional self-information values can indicate redundancy with information represented by the primary’ context data element set. Hence, by¬ selecting a remaining context data element with a greatest conditional self-information value in each iterative round, the computer system can reduce redundancy and preserve context diversity7.5. Evidence Reconstruction
[0128] Optionally at step 608, the computer system can perform an evidence reconstruction process in order to determine whether context compression vectors corresponding to the secondary- context data elements retrain sufficient semantic content information to be useful to the foundation model. The computer system can pass a set of context compression vectors corresponding to the secondary context data elements through a decoder (which can comprise36KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 the foundation model) to reconstruct the secondary context data elements, then compare the reconstructed secondary context data elements to the original secondary context data elements using conditional self-information, token-level overlap, embedding-based similarity, etc., thereby determining whether a RAG framework according to embodiments has learned to interpret and utilize compressed embeddings effectively.
[0129] If the secondary context data elements can be reliably reconstructed from their respective context compression vectors, it implies that those context compression vectors capture their semantic information. In such cases, these context compression vectors can be effectively used in compressed form during generation with the foundation model, reducing context length without compromising answer quality'. Conversely, if reconstruction fidelity' is low, the computer system can fall back to using original natural language version of the secondary’ context data elements to preserve useful information.6. Generate Prompt
[0130] Referring back to FIGs. 1 and 6, at step 610, the computer system can generate a prompt comprising the query and the one or more primary context data elements. In some embodiments, the computer system can generate a prompt comprising the one or more primary’ context data elements, one or more compression vectors corresponding to the one or more secondary context data elements, and the query. In some embodiments, the one or more primary context data elements can be ordered in the prompt based on the determined order. FIG. 1 shows an example prompt in step 128, comprising a preamble 136 (e.g., comprising instructions for foundation model 106), primary context data elements 130, secondary context data elements 132 represented as context compression vectors 134, and the query. FIG. 2 shows a similar prompt template 202.
[0131] In some embodiments, the prompt can comprise a combined token sequence (or can be used to derive a combined token sequence), e.g., a sequence of tokens representative of the information contained in the prompt. As such, in some embodiments, the computer system can tokenize the query, thereby generating a query token sequence corresponding to a token space of the foundation model, e.g., using an embedding or encoding model. The computer system can likewise tokenize the one or more primary context data elements, thereby generating one or more textual context token sequences corresponding to the token space37KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001(e.g., such that each textual context token sequence comprises a sequence of tokens representing a corresponding primary’ context data element). The computer system can then generate the combined token sequence, which can comprise a sequential combination of the one or more textual context token sequences, the one or more compression vectors corresponding to the one or more secondary context data elements, and the query’ token sequence. The one or more textual context token sequences in the combined token sequence can be ordered sequentially based on the determined order. Optionally, the combined token sequence can comprise various other tokens or token sequences, e.g., representative of a textual preamble, headers, etc.7. Generate Query Response
[0132] At step 612, the computer system can generate a query response using the prompt and the foundation model, e g., by inputting the prompt into the foundation model. In some embodiments, the foundation model can be provided by an external model provider server (e.g., an external LLM provider). In such embodiments, the computer system can generate the query response by transmitting the prompt to the external model provider server, such that the external model provider server generates the query response. The external model provider server can then return the query’ response to the computer system (or e.g., provide the query response to a requestor, as described below).
[0133] As described through this disclosure, in some embodiments, the computer system can perform machine learning as a service (MLaaS) for requestors, e.g.. client computers. As such, in some embodiments, the query may comprise a requestor query, the query response may comprise a requestor query response, and the computer system may have received the requestor query from a requestor. As such, in some embodiments, the computer system can cause the requestor query response to be provided to the requestor. In some embodiments, causmg the requestor query response to be provided to the requestor can comprise transmitting the requestor query response to a client computer. In other embodiments, causing the requestor query response to be provided to the requestor can comprise causing an external model provider server to provide the requestor query response to the requestor, e.g., by instructing the external model provider server to transmit the requestor query response to a client computer, e g., after the external model provider server generates the requestor query¬ response using the prompt and the foundation model.38KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001II. EXPERIMENTAL EVALUATION
[0134] Various experiments were conducted, demonstrating the effectiveness of methods according to embodiments and their advantages over existing RAG methods. In general, these experiments involved comparing methods according to embodiments against eight groups of baseline RAG methods spanning three categories. More specifically, the first category of baselines were the ’‘Standard RAG” baselines (Lewis et al. 2020), in which retrieved documents are directly integrated into the input prompt. The second category' of baselines were “compression-based methods”, which condense input passages before feeding them into an LLM, including LLMLingua (Jiang et al. 2023b), LongLLMLingua (Jiang et al., 2024), ICAE (Ge et al., 2024), and xRAG (Cheng et al., 2024). The third category of baselines were “summarization-based methods”, which generated intermediate summaries over retrieved documents to support more focused reasoning, including “Raptor” (Sarthi et al., 2024), “GraphRAG” (Edge et al., 2024), and “InstructRAG” (Wei et al., 2024).
[0135] For summarization-based approaches such as Raptor and GraphRAG, which rely on community -level summarization and long-context reasoning, such summaries can consume too much context, particularly for smaller models with strict context length restrictions (such as Mistral-7B), and as a result such models may struggle with reasoning over long complex inputs and may fail to follow instructions. As such, in some experiments, proprietary models such as GTP-4o (OpenAI, 2025) are used as the base model for summarization questionanswering, following prior work (Luo et al., 2025; Li et al., 2025). Even in such cases, embodiments of the present disclosure outperform these RAG methods.
[0136] In order to demonstrate their modularity7and robustness, the generalizability of embodiments of the present disclosure were evaluated across different retrieval, embedding, and generation components. For a retrieval module, experiments used both sparse and dense retrievers, including “BM25” (Robertson et al., 2004), “bge-reranker-v2-m3” (Li et al., 2023a) and “SFR-Embedding” (Meng et al., 2024).39KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001A. Datasets
[0137] Various datasets were used to evaluate methods and systems according to embodiments, spanning different domains, input lengths, and task types. These domains and task ty pes include short-context question answering, evaluated using the “SQuAD-v2.0” dataset (Rajpurkar et al., 2018) among others. The SQuAD-v2.0 dataset includes questions based on Wikipedia articles, with answers comprising text segments from the corresponding reading passages. For the experiments, questions marked '‘answerable” were used.
[0138] These domains and task types also include long-context question answering, which was evaluated using the “Narrative QA” (Kocisky et al., 2018), QASPER (Dasigi et al., 2021), QuALITY (Pang et al., 2022), and MultifieldQA-en (Bai et al., 2024) datasets. The NarrativeQA dataset is a question-answering dataset based on books and movie transcripts. The QASPER dataset is an information-seeking dataset over scientific research papers with supporting evidence spans. The QuALITY dataset is a reading comprehension benchmark with approximately 5000-token passages and unambiguous questions that require consolidating information from multiple text segments. The MultifieldQA-en dataset is a single document question and answer dataset from diverse sources (arXiV, C4, Wikipedia, WuDaoCorpora, etc.).
[0139] These domains and task types also include multi -hop reasoning, which can involve multi-hop inference between data elements (e.g., documents). This domain / task type was evaluated using the “HotpotQA” (Yang et al., 2018), “TriviaQA” (Joshi et al., 2017) and 2WikiMultihopQA (Ho et al., 2020) datasets. The HotpotQA dataset comprises natural questions that require multi-hop reasoning. The questions are annotated with supporting facts. The TriviaQA dataset comprises trivia questions paired with web evidence (news, encyclopedia, and blogs). The 2WikiMultihopQA dataset is a multi-hop question and answer dataset combining structured and unstructured evidence with reasoning paths.
[0140] These domains and task types also include summarization reasoning, which was evaluated using the “QMSum” dataset (Zhong et al. 2021), among others. These datasets include both multiple-choice and open-ended answer formats, providing a comprehensive testbed for evaluating the generality and adaptability of methods and systems according to embodiments. The QMSum dataset comprises a query-focused meeting-summarization task40KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 spanning several domains, in which models are tasks to locate relevant segments and provide a concise answer from dialog transcripts.
[0141] SQuAD-v2. NarrativeQA, QASPER. QuALITY, HotpotQA, and Trivia QA were used for both training and evaluation. MultifieldQA-en, 2WikiMultihopQA, and QMSum were used for out-of-domain evaluation. All corpora were split into 256-token chunks based on sentence structure. The token-count distribution of the various datasets is presented in FIG. 8 and overall statistics are presented in FIG. 9. To improve fine-tuning, GPT-4o (OpenAI, 2025) was used to convert the fine-tuning dataset into instruction-following format, following previous works (Liu et al., 2023; Xiao et al., 2024b)B. Metrics
[0142] These experiments used evaluation protocols consistent with prior work (Asai et al., 2023; Cheng et al., 2024; Sarthi et al., 2024; Edge et al., 2024). For holistic evaluation and free-form answer generation, lexical metrics, such as ROUGE (R-L) (Lin, 2004) and Fl match scores were used to measure lexical overlap between predicted and ground-truth answers. Four LLM-based metrics (Es et al., 2024) were also used to complement the lexical metrics. These LLM-based metrics capture orthogonal dimensions associated with reliable RAG deployment (Es et al. 2024; Risch et al., 2021), and return a value in the range [0, 1], with higher values indicating better performance. In more detail, a “Faithfulness” metric measured whether an answer generated using an LLM is grounded in the retrieved context, by decomposing answers into atomic claims using an LLM (e.g., GPT-4o). Each claim was then tested for entailment against the retrieved context. Answers fully supported by evidence (e.g., data elements) are favored, and hallucinations are penalized. An “Answer Relevance” (or “Response Relevance”) metric judged how directly the answer addresses queries.Redundant, off-topic, or missing information lowered the score. This metric does not take factual accuracy into consideration. A “Factual Correctness” metric used claim decomposition and natural language inference to verify model generated statements against reference texts. A “Semantic Similarity” metric used a cross-encoder to compute the semantic overlap between generated answers and ground-truth references.41KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001C. Experimental Implementation Details
[0143] Some experiments were based on Pytorch (Paszke et al., 2019), transformers (Wolf et al., 2020), and llama-index (Liu, 2022). Models and data in these experiments used the bfloat16 data type. For LoRA setup, experiments used a rank attention dimension of 16, a scaling factor a = 32, and dropout of 0.1. For chunking, the chunk size was set to 256. Models processed at most n = 10 chunks. In such experiments, the top k = 5 chunks were selected as natural language evidence and the remaining chunks were encoded as compression vectors. To remove stochasticity. the sampling temperature was fixed at zero. Experiments were performed on a Linux server with 6 NVIDIA Al 00 GPUs.D. Performance
[0144] As shown in the Table of FIG. 10, embodiments of the present disclosure (using both conditional self-information (CSI) and embedding-based approaches (EMB) to perform re-ranking) demonstrated considerable improvements over all baselines. For example, on the HotpotQA dataset (related to the “multi-hop reasoning” domain), methods according to embodiments achieved an Fl score of 84.21 and a ROUGE-L score of 78.16, outperforming GraphRAG (Fl: 73.23, R-L: 68.21) by a wide margin. Methods according to embodiments outperformed baseline methods on various other datasets, including QASPER (Fl: 40.55 vs 34.09 for xRAG), NarrativeQA (Fl: 69.46 vs 64.93 for GraphRAG), TriviaQA (Fl: 85.08 vs 77.52 for GraphRAG), and QuALITY (Fl: 52.93 vs 44.13 for ICAE). These results indicate that methods according to embodiments generalize effectively for both single document and multi-document QA tasks.
[0145] The table of FIG. 11 compares the performance of methods according to embodiments and various baseline methods on the SQuAD-v2 dataset. As the sizes of the retrieved data elements are small, LLMs are able to digest the retrieved data elements without exceeding their context limits. Thus, methods according to embodiments only outperform the base models with a small margin. Meanwhile, methods based on summarization decrease performance compared with the finetuned version of the model, as they may reduce the accuracy of (or remove) numerical values or other specific data present in the retrieved data elements.42KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0146] When using smaller models such as Mistral-7B as the base model, long-context RAG baselines such as GraphReader (Li et al., 2024 A), GraphRAG (Edge et al., 2024), and Raptor (Sarthi et al., 2024) exhibit suboptimal performance and often fail to follow instructions. While such methods systematically structure context and generate summaries, their effectiveness is hindered by limited attention spans. This limitation becomes particularly problematic for customizing smaller long-context RAG models. Moreover, community-generated summaries tend to be overly generic, frequently omitting entity names and numeric values, which reduces their utility for queries requiring precise entity recognition or numerical accuracy. In contrast, methods according to embodiments outperform advanced methods like GraphRAG even when GraphRAG uses GPT-4o (OpenAI, 2025) as the base model. This demonstrates the superior contextual understanding and retrieval efficiency associated with methods according to embodiments.
[0147] When contexts are generally short and fit within the base model’s native context window (e.g., for the SQuAD-v2 dataset), methods according to embodiments perform on-par with baseline methods. While methods according to embodiments generally perform better for long-context scenarios, they still maintain strong performance in short context scenarios in which compression is unnecessary.1. Results Under Context Constraints and Impact of Context Budgets
[0148] The table of FIGs. 12-14 compare methods according to embodiments and strong compression-based methods under strict context length constraints (512 and 1024 tokens). Embodiments consistently outperform baselines on both lexical (Fl, ROUGE-L) and LLM-based evaluation metrics. Under 512 tokens, embodiments improve Fl by 19.4% and ROUGE-L by 20.8% on average. Notably, greater improvements can be seen on knowledge-intensive tasks like TriviaQA (+24.5%) and HotpotQA (+29.0%), which require facts and reasoning. Improvements on narrative-style tasks (e.g., NarrativeQA) are more modest, particularly under 1024 tokens (+6.6% Fl and 6.8% ROUGE-L), likely because chunking and compression can change the narrative flow and obscure subtle discourse-level cues. Unlike factoid questions, narrative questions demand holistic coherence that is harder to retain under chunking and summarization (Ge et al.. 2024).43KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001
[0149] Increasing the context budget from 512 to 1024 tokens generally improved performance. Baselines that produce natural language compression (e.g., LongLLMLingua) saw substantial gains (up to +10.6 Fl on NarrativeQA), as the additional budget reduces the need to truncate or overly compress passages, allowing inputs to better reflect their original structure. Embodiments retain a clear performance lead, outperforming the strongest baseline by 6 to 12 Fl on knowledge-intensive tasks such as TriviaQA and HotpotQA. Embodiments show more modest gains on certain datasets (e.g., +4.1 Fl on QASPER).2. Balancing Compression Efficiency and Answer Faithfulness
[0150] One challenge in RAG is balancing compression efficiency with faithfulness.Aggressive approaches like xRAG, which compress entire evidence sets into a single dense vector, optimize for efficiency at the cost of factuality and hallucination. As shown in the table of FIG. 10, baselines like xRAG especially struggle on knowledge-intensive tasks, achieving only 43.4 Fl and 35.5 ROUGE-L on TriviaQA, in contrast to embodiments’ 85.1 Fl and 83.9 ROUGE-L. Quantitative analysis in the table of FIG. 15 reveals that baselines can hallucinate content, generating answers with fabricated entities or tasks ('sentiment analysis’ and 'machine translation’) ungrounded in the original documents. Methods that over-compress inputs (e.g., ICAE) risk discarding critical content. As a result, LLMs tend to become overly conservative and frequently conclude that an answer is not present. These failures underscore the drawbacks of one-shot compression when multiple facts must be retained. In contrast, methods according to embodiments can accurately recover fine-grained content, such as specific task names (e g., NLI, document, and intent classification) prompted in the question with high fidelity7, even under tight context budgets. Thus, embodiments’ hybrid approach preserves salient content, simplifying key information while mitigating factual distortion under tight context budgets.3. Comparison with Summarization-Based Methods
[0151] Embodiments consistently outperform standard RAG and state-of-the-art summarization-based baselines, including Raptor and GraphRAG, despite their use of stronger base models like GPT-4o (OpenAI, 2025) for question-answering and summarization. As shown in the table of FIG. 10, on HotpotQA, which requires multi -hop reasoning, embodiments achieve +15% Fl and +14.6% ROUGE-L. These results highlight44KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 the effectiveness of compression approaches according to embodiments, which help foundation models (e.g., LLMs) accommodate and reason over multiple discrete evidence pieces within constrained context windows.4. Performance on Short-Context QA
[0152] The SQuAD-v2 QA task presented minimal challenge in context length, as each query is paired with a single passage that fits within the model's input window in most cases. Accordingly, the performance gap between models narrows. As shown in the table of FIG.11, embodiments (labeled " SARA") achieve the highest results (76.55 Fl, 69.22 ROUGE-L), outperforming the strongest baselines by a modest margin (+3.98 Fl, +2.19 ROUGE-L). In contrast, aggressively compressed systems such as xRAG and ICAE perform significantly worse (<60.19 Fl), likely due to summaries that obscures critical details (such as entity names, numeric values, and specific events), reducing accuracy even when full text fits into the model.5. Generalization Across Model Architectures and Sizes
[0153] Experiments were performed to evaluate whether RAG methods according to embodiments achieved performance improvements for various different LLMs. Aside from Mistral-7B, experiments were performed to evaluate embodiments on four additional models from three model families (Mistral, Llama, and Gemma) spanning various sizes and architectures, i.e., MistralNemo-12B, MistralSmall-24B, Llama3.1-8B, and Gemma3-4B. As shown in FIGs. 16 and 17, Embodiments consistently outperform the baseline, with up to a +40 improvement in Answer Relevance, +14 in Answer Correctness, and +21 in Semantic Similarity. Improvements were particularly pronounced on smaller models. On Mistral-7B, embodiments improve answer relevance by 17.71. answer correctness by 13.72, and semantic similarity by 15.53. These results highlight embodiments’ ability to optimize context usage under tighter context budgets, making them especially effective for smaller models. In some cases, embodiments enabled a 7B model to match or surpass a much larger one (e.g., MistralSmall-24B). highlighting that reasoning over mixed-format contexts can close the performance gap without increasing model sizes.
[0154] In general, performance gains were greater when a compression model and LLM shared the same architecture (e.g., Mistral). Among the Mistral family, an average boost in45KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Answer Relevance of 20.12 and Answer Correctness of 7.07 were observed. MistralNemo and Mistral Small achieved improvements in response relevance of +19.65 and +23.01. and semantic similarity of +20.44 and +14.38 respectively. This suggests that architectural alignments between the compression models and LLMs enhanced semantic compatibility between compressed inputs and answer generation. In contrast, Gemma-3 showed modest gains (+6.83 in answer relevance and +5.82 in answer correctness), likely due to its architectural mismatch.
[0155] A strength of embodiments is their ability to refine and reorganize retrieved context to support finer-grained reasoning. Since both embodiments and RAG baselines leveraged the same initial retriever, they operated over comparable evidence. As a result, faithfulness, i.e., factual consistency with the retrieved context, shows modest improvements.6. Generalization Across Retrievers
[0156] Embodiments were evaluated using dense retrievers like multi-qa-mpnet-base-cos-vl (Song et al., 2020) and SFR (Meng et al., 2024) in addition to BM25 (Robertson et al., 2004). As shown in the table of FIG. 18, embodiments perform consistently across retrievers, confirming their model-agnostic design. Dense retrievers, especially SFR, yield stronger results (e.g., achieving +10 Fl over BM25 on QASPER), highlighting the value of semantically richer base retrievers for complex, multi-hop QA. Overall, embodiments remain robust to retriever choice while benefiting from higher quality evidence.7. Ablation Studies
[0157] To quantify the contribution of various aspects of methods according to embodiments (e.g., context compression, reconstruction, and reranking), three variant methods according to embodiments were evaluated, labeled “SARA-C’’, “SARA-P”, and “SARA-R”. SARA-C removes compression vectors and only processes contextual data in natural language format. SARA-P removes context reconstruction objectives during training. SARA-R skips the adaptive reranking stage, relying solely on initial BM25 retrieval.
[0158] As evident from FIG. 19, removing the reconstruction objective (SARA-P) results in the most substantial performance drop of -7 to -9 Fl across all datasets. This confirms that learning to reconstruct full contexts from compression vectors is essential for preserving semantics and leveraging these vectors for answer generation. Further, disabling46KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 compression (SARA-C) leads to consistent performance declines, especially on TriviaQA (-5.6 Fl) where the long form contexts are potentially noisy or irrelevant. Compression helps filter salient content and suppress redundancy, enhancing answer correctness. Removing reranking (SARA-R) yielded modest but consistent drops, confirming that compression-aware reranking methods according to embodiments improve evidence selection beyond lexical similarity at minimal computational cost, especially when initial retrieval is suboptimal.8. Sensitivity Analysis
[0159] Sensitivity analysis experiments were conducted to test whether methods according to embodiments can effectively leverage compressed context. The total number of retrieved data elements was fixed to N = 10 per query and the number of top-ranked natural language data elements k was varied. As shown in FIG. 20, performance remained strong even with minimal natural language input (e.g., for k = 1, Fl = 38.54 and ROUGE-L = 39.89), indicating that compression vectors retain essential information. Performance improved with larger k, but plateaued around k = 8 (Fl = 41.6, ROUGE-L = 43.12), and slightly drops at k = 9, suggesting diminishing returns or noise from excessive natural language content. These results highlight the effectiveness of embodiments’ hybrid approach to balance context utility, informativeness, and efficiency.
[0160] To further illustrate such effects, the table of FIG. 21 shows how increasing k within a specific range improves factual specificity. With only compressed context (0 / 10), the model is able to identify a single entity name (CoNLL-2003), whereas increasing k = 5 enables the model to produce answers with high fidelity. Our hybrid approach allows for such precision without overwhelming the context budget.9. Generalizations on Additional Embedding Models
[0161] For embedding alignment experiments, a curriculum learning strategy was used, starting with shorter sentences and gradually transitioning into complex examples.Specifically, spaCy was used for named entity recognition (NER) and to rank sentences by token count and the number of named entities in categories such as PER, ORG, LOC, GPE, Date, Time, and Event. The compressor embedding models used in these experiments are depicted in the table of FIG. 22. These include Salesforce / SFR-Embedding-Mistral (SFR),47KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Linq-AI-Research / Linq-Embed-Mistral (Linq) embedding (Kim et al., 2024b), Alibaba-NLP / gte-Qwen2-7B-instruct (GTE) (Li et al., 2023b), and NovaSearch / stella_en_1.5B_v5 (Stella). FIG. 23 shows the effects of different compression models on Fl and ROUGE-L scores for QASPER, NarrativeQA, and TriviaQA.10. Generalization on Unseen Datasets
[0162] Further experiments evaluate generalization to unseen datasets by testing fine-tuned LLM models on three out-of-domain datasets from LongBench (Bai et al., 2024):MultiFieldQA-en, 2WikiMultihopQA, and QMSum, which differ in domain and task format from the training data (as described above in the Datasets Section). As shown in the table of FIG. 24, embodiments (labeled ‘“SARA”) consistently improve performance across all benchmarks. Embodiments improve Response Relevance by large margins (+18.5 on QMSum, +47.7 on MultifieldQA-en, and +55.0 on 2WikiMultiHopQA). These gains highlight the strength of combining natural language spans with compression vectors, which enable embodiments to leverage more relevant evidence despite domain shifts.Improvements are particularly pronounced on QA-style tasks, suggesting that QA data in the fine-tuning dataset contributes to embodiments’ performance on other QA datasets.Improved relevance also leads to cleaner answers and less hallucinations and off-topic content.
[0163] In contrast, Answer Correctness rises more modestly (+0.3 to +2.2), suggesting that while retrieval quality generalizes well, reasoning over the retrieved content might be partially domain dependent. For example, TriviaQA and QASPER (which were used in training) are based on Wikipedia and academic literature, respectively, while MultiFieldQA-en (used in out-of-domain experiments) involves answering questions based on articles from multiple domains. In this case, in-domain adaptation or instruction tuning could help further improve performance.11. Expressivity of Compression Vectors
[0164] In order to serve as reliable contextual information, compression vectors preferably faithfully represent any underlying semantic information. To evaluate this, experiments were performed in which compression vectors were decoded into natural language and compared against their respective contextual sources. Representative successes for both chunk-level48KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 and paragraph-level reconstruction are shown in the tables of FIGs. 25 and 26. As depicted in FIG. 27, while the length of decoded texts is reasonably correlated with the length of reference texts, decoded texts are usually shorter and serve as higher level summarizations of their respective input contexts. In most cases, the decoded texts preserved core propositions, causal links, and sentiment. Embodiments are able to recover key information, such as exact entities and numeric values (e.g., ‘220'). Losses are mostly fine-grained, e.g., exact dates (‘1903’ to ‘1900s') or numerical magnitudes (‘3400 years' to ‘over 3.000 years') may be paraphrased or omitted. When contexts are longer, the risk of recovery failure is higher. Notably, decoders according to embodiments rarely invent new facts, and missing details are ty pically dropped rather than hallucinated. This behavior implies that compression vectors according to embodiments encode stable, high-level meaning while suppressing fewer specifics, which is a valuable feature for knowledge intensive tasks that demand both factual precision and robust hallucination control.III. COMPUTER SYSTEM
[0165] Any of the computer systems mentioned herein may utilize any suitable number of subsystems. Examples of such subsystems are shown in FIG. 28 in computer system 2800. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be the components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components. A computer system can include desktop and laptop computers, tablets, mobile phones and other mobile devices.
[0166] The subsystems shown in FIG. 28 are interconnected via a system bus 2812.Additional subsystems such as a printer 2808, keyboard 2818, storage device(s) 2820, monitor 2824 (e.g., a display screen, such as an LED), which is coupled to display adapter 2814, and others are shown. Peripherals and input / output (I / O) devices, which couple to I / O controller 2802, can be connected to the computer system by any number of means known in the art such as input / output (I / O) port 2816 (e.g., USB, FireWire®). For example, I / O port 2816 or external interface 2822 (e.g. Ethernet, Wi-Fi, etc.) can be used to connect computer system 2800 to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus 2812 allows the central processor 2806 to communicate with each subsystem and to control the execution of a plurality of instructions from system49KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 memory 2804 or the storage device(s) 2820 (e g., a fixed disk, such as a hard drive, or optical disk), as well as the exchange of information between subsystems. The system memory 2804 and / or the storage device(s) 2820 may embody a computer readable medium. Another subsystem is a data collection device 2810, such as a camera, microphone, accelerometer, and the like. Any of the data mentioned herein can be output from one component to another component and can be output to the user.
[0167] A computer system can include a plurality of the same components or subsystems, e.g., connected together by external interface 2822, by an internal interface, or via removable storage devices that can be connected and removed from one component to another component. In some embodiments, computer systems, subsystems, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components. In various embodiments, methods may involve various numbers of clients and / or servers, including at least 10, 20, 50, 100, 200, 500, 1,000, or 10,000 devices. Methods can include various numbers of communication messages between devices, including at least 100, 200, 500, 1,000, 10,000, 50,000, 100,000, 500,00, or one million communication messages. Such communications can involve at least 1 MB, 10 MB, 100 MB, 1 GB, 10 GB, or 100 GB of data.
[0168] Aspects of embodiments can be implemented in the form of control logic using hardware circuitry (e g., an application specific integrated circuit or field programmable gate array) and / or using computer software stored in a memory with a generally programmable processor in a modular or integrated manner, and thus a processor can include memory storing software instructions that configure hardware circuitry, as well as an FPGA with configuration instructions or an ASIC. As used herein, a processor can include a single-core processor, multi-core processor on a same integrated chip, or multiple processing units on a single circuit board or networked, as well as dedicated hardware. The computations can be performed in parallel by the different processing units and / or different processing threads of a single processing unit. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and / or methods to implement50KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 embodiments of the present disclosure using hardware and a combination of hardware and software.
[0169] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer readable medium for storage and / or transmission. A suitable non-transitory computer readable medium can include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk) or Blu-ray disk, flash memory, and the like. The computer readable medium may be any combination of such devices. In addition, the order of operations may be re-arranged. A process can be terminated when its operations are completed but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.
[0170] Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and / or wireless networks conforming to a variety of protocols, including the Internet. As such, a computer readable medium may be created using a data signal encoded with such programs. Computer readable media encoded with the program code may be packaged with a compatible device (e.g., as firmware) or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g. a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer or other suitable display for providing any of the results mentioned herein to a user.
[0171] Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps. Any operations performed with a processor may be performed in real-time. The term "real-time" may refer to computing operations or processes that are completed within a51KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 certain time constraint. As examples, a time constrain may be 30 seconds, 1 minute, 10 minutes, 30 minutes, 1 hour, 4 hours, 1 day, or 7 days. Thus, embodiments can involve computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing respective steps or a respective group of steps. Although presented as numbered steps, steps of methods herein can be performed at a same time or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, and of the steps of any of the methods can be performed with modules, circuits, or other means of a system for performing these steps.
[0172] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of embodiments of the invention.However, other embodiments of the invention may be directed to specific embodiments relating to each individual aspect, or specific combinations of these individual aspects.
[0173] The above description of example embodiments of the present disclosure has been presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the teaching above.
[0174] A recitation of"an" or 'The” is intended to mean ‘‘one or more” unless specifically indicated to the contrary. The use of “or” is intended to mean an “inclusive or,” and not an “exclusive or” unless specifically indicated to the contrary.. Reference to a “first” component does not necessarily require that a second component be provided. Moreover, reference to a “first” or a “second” component does not limit the referenced component to a particular location unless expressly stated. The term “based on” is intended to mean “based at least in part on.”
[0175] The claims may be drafted to exclude any element which may be optional. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely”, “only”, and the like in connection with the recitation of claim elements, or the use of a “negative” limitation.
[0176] All patents, patent applications, publications and description mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted as prior art.52KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Where a conflict exists between the instant application and a reference provided herein, the instant application shall dominate.53KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001IV. REFERENCESAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In ICLR, 2023Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu. Aohan Zeng. Lei Hou. et al. Longbench: A bilingual, multitask benchmark for long context understanding. In ACL, pages 3119-3137, 2024.Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, and Dongyan Zhao, xrag: Extreme context compression for retrieval-augmented generation with one token. arXiv:2405.13792, 2024.Alexis Chevalier. Alexander Wettig, Anirudh Ajith, and Danqi Chen. Adapting language models to compress contexts. In EMNLP, pages 3829-3846, 2023.Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. A dataset of information-seeking questions and answers anchored in research papers. In NAACL, pages 4599-4610, 2021.Jiayu Ding. Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, and Furu Wei. Longnet: Scaling transformers to 1,000,000,000 tokens. arXiv: 2307.02486, 2023.Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. From local to global: A graph rag approach to query- focused summarization. arXiv: 2404.16130, 2024.Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. Ragas: Automated evaluation of retrieval augmented generation. In EACL, pages 150-158. 2024.Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. Enabling large language models to generate text with citations. In EMNLP, pages 6465-6488. ACL, 2023.Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. In ICLR, 2024.Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In ICLR, 2021.Gautier Izacard, Patrick Lewis. Maria Lomeli, Lucas Hosseini, Fabio Petroni. Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: Few-shot learning with retrieval augmented language models. JMLR, 24(251): 1-43, 2023.Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Longllmlingua: Accelerating and enhancing 11ms in long context scenarios via prompt compression. In ACL, 2024.54KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Hongye Jin, Xiao tian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. Llm maybe longlm: Selfextend 11m context window without tuning. In ICML, pages 22099-22114, 2024.Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In ACL, pages 1601-1611, 2017.Jang-Hyun Kim, Junyoung Yeom, Sangdoo Yun, and Hyun Oh Song. Compressed context memory for online language model interaction. In ICLR, 2024a.Junseong Kim, Seolhwa Lee, Jihoon Kwon, Sangmo Gu, Yejin Kim, Minkyung Cho, Jy- yong Sohn, and Chanyeol Choi. Linq-embed-mistral: elevating text retrieval with improved gpt data through task-specific control and quality refinement. Linq Al Research Blog, 2024b. URL https: / / getlinq.com / blog / linq-embed-mistral / .Tomas Kocisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328, 2018.Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rocktaschel, et al.Retrieval-augmented generation for knowledge-intensive nip tasks. NeurlPS, 33:9459-9474, 2020.Chaofan Li, Zheng Liu, Shitao Xiao, and Yingxia Shao. Making large language models a better foundation for dense retrieval. arXiv:2312.15503, 2023a.Shilong Li, Yancheng He, Hangyu Guo, Xingxuan Bu, Ge Bai, Jie Liu, Jiaheng Liu, Xingwei Qu, Yangguang Li, Wanli Ouyang, et al. Graphreader: Building graph-based agent to enhance long-context abilities of large language models. In EMNLP, pages 12758-12786, 2024a.Wenyan Li. Jiaang Li, Rita Ramos, Raphael Tang, and Desmond Elliott. Understanding retrieval robustness for retrieval-augmented image captioning. In ACL, pages 9285- 9299. 2024b.Zehan Li. Xin Zhang, Yanzhao Zhang. Dingkun Long, Pengjun Xie, and Meishan Zhang.Towards general text embeddings with multi-stage contrastive learning. arXiv: 2308.03281, 2023b.Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. pages 74-81. 2004.Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurlPS, 36: 34892-34916, 2023.Jerry Liu. Llamalndex, 11 2022. URL https: / / github.com / jerryjliu / llama_index.55KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Shudong Liu, Yiqiao Jin, Cheng Li, Derek F Wong, Qingsong Wen, Lichao Sun, Haipeng Chen, Xing Xie, and Jindong Wang. Culturevlm: Characterizing and improving cultural understanding of vision-language models for over 100 countries. arXiv: 2501.01282, 2025.Rui Meng, Ye Liu, Shafiq Ray han Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz.Sfr- embedding-mistral: enhance text retrieval with transfer learning. Salesforce Al Research Blog, 2024. URL https: / / www.salesforce.com / blog / sfr-embedding / .Jesse Mu, Xiang Li, and Noah Goodman. Learning to compress prompts with gist tokens.NeurlPS, 36:19327-19352. 2023.Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. In ACL, pages 2014-2037, 2023.Tsendsuren Munkhdalai, Manaal Faruqui, and Siddharth Gopal. Leave no context behind:Efficient infinite context transformers with infini-attention. arXiv: 2404.07143, 2024.OpenAI. Gpt-4o, 2025. URL https: / / chat.openai.com / .Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rtihle, Yuqing Yang, Chin-Yew Lin, et al. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. In ACL, pages 963-981, 2024.Richard Yuanzhe Pang, Alicia Parrish. Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, et al. Quality': Question answering with long input texts, yes! In NAACL. pages 5336-5358, 2022.Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury7, Gregory7Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In NeurlPS, volume 32, 2019.Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for squad. In ACL, pages 784-789, 2018.Julian Risch, Timo Moller, Julian Gutsch, and Malte Pietsch. Semantic answer similarity for evaluating question answering models. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 149-157, 2021.Stephen Robertson, Hugo Zaragoza, and Michael Taylor. Simple bm25 extension to multiple weighted fields. In CIKM, pages 42-49, 2004.Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D.Manning. Raptor: Recursive abstractive processing for tree-organized retrieval. In ICLR. 2024.56KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Claude E Shannon. A mathematical theory of communication. The Bell system technical journal, 27 (3):379-423, 1948.Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu. and Tie-Yan Liu. Mpnet: Masked and permuted pre-training for language understanding. NeurlPS, 33: 16857-16867, 2020.Zhepei Wei, Wei-Lin Chen, and Yu Meng. Instructrag: Instructing retrieval-augmented generation via self-synthesized rationales. In ICLR, 2025.Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac. Tim Rault, Remi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. In EMNLP, pages 38-45, 2020.Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Haifeng Chen, et al. Large language models can be good privacy¬ protection learners. In EMNLP, 2024.Fangyuan Xu, Weijia Shi, and Eunsol Choi. Recomp: Improving retrieval-augmented 1ms with compression and selective augmentation. In ICLR, 2024.Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In EMNLP, pages 2369-2380, 2018.Chanwoong Yoon, Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, and Jaewoo Kang.Compact: Compressing retrieved documents actively for question answering. In EMNLP, 2024.Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval-augmented generation in 11ms. In NeurlPS, 2024.Qianchi Zhang, Hainan Zhang, Liang Pang, Hongwei Zheng, and Zhiming Zheng. Adacomp:Extractive context compression with adaptive predictor for retrieval-augmented large language models. arXiv: 2409.01579, 2024a.Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, and Zhangyang Wang. Found in the middle: How language models use long contexts better via plug-and-play positional encoding. In NeurlPS, 2024b.Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu. Hao Chen, and Xing Xie.Competeai: Understanding the competition behaviors in large language modelbased agents. In ICML, 2024.Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, et al. Qmsum: A new benchmark for query-based multi- domain meeting summarization. In NAACL, pages 5905-5921, 2021.57KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001Wenhao Zhu, Pinzhen Chen, Hanxu Hu, Shujian Huang, Fei Yuan, Jiajun Chen, and Alexandra Birch. Generalizing from short to long: Effective data synthesis for long-context instruction tuning. arXiv: 2502.15592. 2025.58KILPATRICK TOWNSEND 156013684 1
Claims
PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001WHAT IS CLAIMED IS:
1. A method for generating a query response corresponding to a query using a foundation model and a plurality of context data elements, the method performed by a computer system and comprising:identifying the plurality of context data elements based on relevance to the query;generating a plurality of context compression vectors corresponding to the plurality of context data elements;determining, based on the plurality of context compression vectors, one or more primary context data elements of the plurality of context data elements, a determined order of the one or more primary context data elements, and one or more secondary context data elements of the plurality of context data elements, wherein the determined order is based on relevance of each of the one or more primary7context data elements to the query7;generating a prompt comprising the one or more primary context data elements, one or more compression vectors corresponding to the one or more secondary context data elements, and the query, wherein the one or more primary context data elements are ordered in the prompt based on the determined order; andgenerating the query7response using the prompt and the foundation model.
2. The method of claim 1, wherein the query comprises a requestor query, wherein the query response comprises a requestor query response, and wherein the method further comprises:receiving the requestor query from a requestor; andcausing the requestor query response to be provided to the requestor.
3. The method of claim 2, wherein the requestor comprises a client computer, and wherein causing the requestor query7response to be provided to the requestor comprises transmitting the requestor query response to the client computer.
4. The method of claim 2, wherein generating the query7response using the foundation model comprises transmitting the prompt to an external model provider server, wherein the external model provider server generates the query response using the prompt and the foundation model.59KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W0015. The method of claim 4, wherein causing the query response to be provided to the requestor comprises causing the external model provider server to transmit the query response to the requestor.
6. The method of claim 1, wherein the plurality of context data elements comprise a plurality of digital documents and / or chunks of digital documents stored in a database, and wherein identifying the plurality of context data elements comprises searching a database of context data elements using a retriever module.
7. The method of claim 1, wherein the plurality of context compression vectors corresponds to a token space of the foundation model.
8. The method of claim 1, wherein the prompt comprises a combined token sequence, and wherein generating the prompt comprises:tokenizing the query, thereby generating a query token sequence corresponding to a token space of the foundation model;tokenizing the one or more primary context data elements, thereby generating one or more textual context token sequences corresponding to the token space; and generating the combined token sequence, wherein the combined token sequence comprises a sequential combination of the one or more textual context token sequences, the one or more compression vectors corresponding to the one or more secondary¬ context data elements, and the query token sequence, wherein the one or more textual context token sequences in the combined token sequence are ordered sequentially based on the determined order.
9. The method of claim 1, wherein the plurality of context compression vectors is generated using a compression model, wherein the method further comprises training the compression model to generate context compression vectors in a training process comprising:generating a plurality of training context compression vectors corresponding to a plurality of training context data elements using the compression model;decoding the plurality of training context compression vectors using the foundation model, thereby generating a plurality of output training context data elements;60KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 determining an embedding alignment loss based on a difference between the plurality’ of training context data elements and the plurality- of output training context data elements;generating a plurality of training context data sequences, each training context data sequence comprising one or more training context data elements;generating a plurality of training context compression vector sequences corresponding to the plurality of training context data sequences;decoding the plurality of training context compression vector sequences using the foundation model, thereby generating a plurality- of output training context data sequences;determining a context reconstruction loss based on a difference between the plurality of training context data sequences and the plurality of output training context data sequences; andupdating parameters of the compression model based on the embedding alignment loss and the context reconstruction loss, thereby training the compression model.
10. The method of claim 1, wherein the foundation model comprises a pretrained foundation model, and wherein the foundation model is instruction-tuned to generate query responses to prompts using a low-rank adaptation module.
11. The method of claim 1, wherein a number of pri maty context data elements in the one or more primary- context data elements is equal to a predetermined primary context count, and wherein a number of secondary context data elements in the one or more secondary context data elements is equal to a predetermined secondary context count.
12. The method of claim 1, wherein determining the one or more primarycontext data elements, the determined order of the one or more primary context data elements, and the one or more secondary context data elements comprises:identifying a first context data element from the plurality- of context data elements based on a relation of the first context data element to the query;generating a primary context data element set initially comprising the first context data element; and61KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 performing one or more iterative rounds, wherein each iterative round comprises updating the primary’ context data element set to include a selected context data element from one or more remaining context data elements of the plurality of context data elements, thereby generating an updated primary' context data element set, wherein each selected context data element is selected based on a comparison of each remaining context data element of the one or more remaining context data elements to the primary context data element set, wherein the one or more primary context data elements comprise the updated primary context data element set after the one or more iterative rounds are performed, wherein the determined order comprises an order in which each selected context data element was added to the primary context data element set, and wherein the one or more secondary¬ context data elements comprise one or more unselected context data elements from the plurality of context data elements that are not in the primary context data element set.
13. The method of claim 12, further comprising generating a query- vector corresponding to the query, wherein the comparison of each remaining context data element to the primary context data element set is an embedding-based comparison that comprises minimizing a difference between the query vector and a comparison vector corresponding to the updated primary- context data element set, wherein the comparison vector comprises a combination of context compression vectors corresponding to the one or more primary context data elements in the updated primary- context data element set.
14. The method of claim 12, wherein the comparison of each remaining context data element to the primary- context data element set comprises a conditional selfinformation comparison that comprises maximizing self-information of a remaining context data element given the primary- context data element set, wherein the selected context data element comprises the remaining context data element with maximized self-information.
15. A method for generating a query response corresponding to a query using a foundation model and a plurality of context data elements, the method performed by a computer system and comprising:identifying a plurality of context data elements based on relevance to the query;62KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 identifying a first context data element from the plurality of context data elements based on a relation of the first context data element to the query;generating a primary context data element set initially comprising the first context data element;performing one or more iterative rounds, wherein each iterative round comprises updating the primary' context data element set to include a selected context data element from one or more remaining context data elements of the plurality of context data elements, thereby generating an updated primary' context data element set, wherein each selected context data element is selected based on a comparison of each remaining context data element of the one or more remaining context data elements to the primary context data element set;generating a prompt comprising the query and a plurality' of primary context data elements in the primary context data element set; andgenerating the query response using the prompt and the foundation model.
16. The method of claim 15, wherein identifying the plurality of context data elements comprises:generating a query vector corresponding to the query; andperforming a context data element search using a retriever module by comparing the query vector to data element vectors representative of a plurality of data elements, wherein the plurality of context data elements comprise a plurality of relevant context data elements from among the plurality of data elements according to the context data element search, and wherein the first context data element comprises a most relevant context data element according to the context data element search.
17. The method of claim 15, yvherein the method further comprises generating a query vector corresponding to the query and generating a plurality of context compression vectors corresponding to the plurality of context data elements, and wherein updating the primary context data element set to include the selected context data element from the one or more remaining context data elements, thereby generating the updated primary' context data element set comprises:generating a candidate primary context data element set for each remaining context data element, thereby generating one or more candidate primary context data element63KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 sets, wherein each candidate primary context data element set comprises the primary context data element set and a different candidate context data element;generating a candidate set comparison vector for each candidate primary context data element set, thereby determining one or more candidate set comparison vectors, wherein each candidate set comparison vector comprises a combination of context compression vectors corresponding to one or more context data elements in that candidate primary context data element set; andevaluating the one or more candidate primary context data element sets by comparing each candidate set comparison vector to the query vector, wherein the updated primary context data element set comprises a candidate primary context data element set for which a corresponding comparison vector is most similar to the query vector, and wherein the selected context data element comprises the different candidate context data element included in that candidate primary context data element set.
18. The method of claim 17, wherein similarity between comparison vectors and the query vector is determined based on a Euclidean distance between each comparison vector and the query vector.
19. The method of claim 15, wherein updating the primary context data element set to include the selected context data element from the one or more remaining context data elements, thereby generating the updated primary’ context data element set comprises:determining, for each remaining context data element, a conditional selfinformation value, thereby determining one or more conditional self-information values;determining a greatest conditional self-information value;identifying the selected context data element based on the greatest conditional self-information value, wherein the selected context data element comprises a remaining context data element corresponding to the greatest conditional self-information value; and adding the selected context data element to the primary' context data element set, thereby' generating the updated primary context data element set.
20. A computer system comprising:one or more processors; and64KILPATRICK TOWNSEND 156013684 1PATENTAttorney Docket No.: 079900-1550464Client Reference No.: 10538W001 a non-transitory computer readable medium coupled to the one or more processors, the non-transitory computer readable medium comprising instructions that, when executed by the one or more processors, cause the computer system to perform the method of any of claims 1-19.65KILPATRICK TOWNSEND 156013684 1
