Method for retrieving context extraction defenses in large language models for retrieval augmentation generation of augmentations

Abstract

A computer-implemented method for defending against context extraction in a large language model generating augmentations by retrieving augmentations, the method comprising: computing an optimized proxy context embedding for an original context embedding, the optimized proxy context embedding minimizing a difference between a response to a regular query generated by the large language model based on the original context embedding and a response to the regular query generated by the large language model based on the optimized proxy context embedding, and maximizing a difference between a response to an attack query generated by the large language model based on the optimized proxy context embedding and the original context; and generating, by the large language model, a response to an input query based on the optimized proxy context embedding.

Description

Technical Field

[0001] This disclosure relates to a method for retrieving context extraction defenses in large language models that enhance generative reinforcement. Background Technology

[0002] Retrieval-Enhanced Generation (RAG) refers to a method in Large Language Modeling (LLM) that combines information retrieval with text generation. In this approach, the model first retrieves relevant information from an external knowledge base or corpus, and then this information is used to inform and enhance subsequent responses. By combining retrieval-based techniques with generative capabilities, RAG improves the accuracy, relevance, and realism of the model's output, particularly for tasks requiring specialized or up-to-date knowledge.

[0003] This disclosure deals with RAG settings, where the retrieved context should be protected and inaccessible to RAG users. This is typically required when a RAG-enhanced LLM is provided externally as a service by a private or corporate entity. RAG providers may use privacy-related or confidential information to establish a grounded RAG generation phase in order to improve the overall performance of the system or service offered.

[0004] Existing RAG context extraction defenses primarily focus on query manipulation, retrieval constraints, and system directives. However, these methods suffer from fundamental weaknesses. Adjusting the placement of in-prompt queries or modifying retrieval thresholds often introduces instability, degrading response quality and failing to guarantee robust protection. Similarly, system-level directives that restrict content disclosure remain vulnerable to sophisticated adversarial queries specifically designed to override such constraints.

[0005] Given these shortcomings, a more resilient defense mechanism is needed to protect the knowledge base while preserving response quality and availability. Summary of the Invention

[0006] According to a first aspect of this disclosure, this objective is achieved by means of a computer-implemented method for defending against context extraction in a large language model enhanced through retrieval enhancement, the method comprising: - Optimized proxy context embedding for the original context embedding, wherein the optimized proxy context embedding is: - Minimize the difference between the responses to regular queries generated by the large language model based on the original context embedding and the responses to regular queries generated by the large language model based on the optimized proxy context embedding, and - Maximize the difference between the response to the attack query generated by the large language model based on optimized proxy context embedding and the original context, and - Based on optimized proxy context embedding, the response to the input query is generated by the large language model, that is, without passing the original context or original context embedding to the large language model.

[0007] The utility of Retrieval Enhancement Generation (RAG)-enhanced Large Language Model (LLM) is preserved by minimizing the difference between the response to a regular query generated by the Large Language Model based on the original context embedding and the response to a regular query generated by the Large Language Model based on the optimized proxy context embedding. Original context extraction is prevented by maximizing the difference between the response to an attack query generated by the Large Language Model based on the optimized proxy context embedding and the original context.

[0008] The computation of optimized proxy context embedding can be performed through the following: - Minimize the cross-entropy loss between the response to the predetermined query set representing the original context, generated by the large language model based on the original context embedding, and the response to the predetermined query set, generated by the large language model based on the proxy context embedding. - Minimize the cross-entropy loss between the response to a fixed query and the fixed target context generated by the large language model based on the proxy context embedding.

[0009] For example, the optimized agent context embedding can be computed by solving the following optimization problem: .

[0010] here, Indicates the original context, Indicates a pre-defined query set. Represents query set The query, Indicates query Embedded, Indicates the embedding of the original context. Indicates a fixed query Embedded, This indicates that the proxy context is embedded. This indicates a fixed target context, and This represents the cross-entropy loss.

[0011] The queries in this query set can be synthesized using a large language model that references the original context. Fixed queries are proxy queries for unknown attack queries. Fixed queries may be trivial attack strategies and may not be intended for attack success. Instead, they may only be used in the optimizer and may serve as a lower bound for potential attack queries. The fixed target context can be specified to be semantically different from the original context.

[0012] The optimization objective above is solvable. However, directly computing the optimized proxy context embedding for each original context in large datasets can be computationally complex. To scale to large data storage, the computation of the optimized proxy context embedding can be approximated by a proxy generator neural network, which can be trained using a training method including: - Provides a primitive context dataset that includes multiple primitive contexts; - Calculate the corresponding original context embedding for each original context in the context dataset; - Using the method described above, compute the corresponding optimized proxy context embedding for each original context embedding; - Provides training data including pairs of original context embeddings and corresponding proxy context embeddings; - Train the agent generator neural network to learn to map the original context embedding to the optimized agent context embedding.

[0013] Training can be achieved by solving the following: , in This represents the parameters of the proxy generator neural network. Represents the original context dataset. Represents the original context dataset The original context, This represents a proxy generator neural network. This indicates the original context embedding, and This indicates an optimized proxy context embedding.

[0014] Once trained It can generate proxy embeddings in a single forward pass, eliminating the need for per-context optimization. For key or outlier contexts where the proxy generator's predictions underperform (e.g., low-confidence embeddings detected via outlier scores), a lightweight fine-tuning step can refine the proxy context embeddings using the following expression: , The expression was discussed above.

[0015] In this way, robustness can be ensured without sacrificing scalability.

[0016] By solving the following: , The training of the proxy generator model can also be combined with the above expression: .

[0017] here, This represents a dataset that includes pairs of contexts and the queries corresponding to those contexts. Indicates the original context, Indicates a query. Indicates query Embedded, Indicates the embedding of the original context. This indicates the embedding of a fixed query. This represents a mapping of the original context embedded by the proxy generator neural network. This indicates a fixed target context, and This represents the cross-entropy loss.

[0018] In a second aspect of this disclosure, a data processing apparatus is provided, the data processing apparatus including components configured to perform the methods described above.

[0019] In a third aspect of this disclosure, a computer program including instructions is provided that, when executed by a computer, cause the computer to perform the previously described method.

[0020] In a fourth aspect of this disclosure, a computer-readable storage medium is provided that includes instructions which, when executed by a computer, cause the computer to perform the previously described methods. Attached Figure Description

[0021] This disclosure will be described in detail below with reference to the accompanying drawings. In the drawings: Figure 1 This is a schematic diagram illustrating an example of a RAG-enhanced LLM.

[0022] Figure 2 This is a flowchart of an exemplary method according to this disclosure.

[0023] Figure 3 This is a schematic diagram of an exemplary data processing apparatus according to the present disclosure. Detailed Implementation

[0024] Figure 1 This is a schematic diagram illustrating an example of a large language model (LLM) enhanced by retrieval-enhanced generation (RAG), which will be referred to hereafter as RAG system 100.

[0025] Retrieval-enhanced generation (RAG) refers to a method that combines information retrieval with text generation in a large language model. In this method, when a user query 110 is received, the RAG system 100 retrieves relevant information 120 from an external knowledge base or corpus. The user query 110, the retrieved information 120, and the system prompt 130 are synthesized into a coherent prompt, and the large language model 140 has the task of developing a response 150 to this coherent prompt. By combining retrieval-based techniques with generative capabilities, RAG improves the accuracy, relevance, and realism of the model output, especially for tasks requiring specialized or up-to-date knowledge.

[0026] In RAG (Relational Access Registry) settings, privacy-related or confidential information is used to establish the RAG generation phase, and the retrieved context should be protected and inaccessible to RAG users. This is particularly relevant to RAG context extraction attacks, which target RAG-enhanced systems to reconstruct or expose proprietary knowledge bases, document repositories, or context data used for response generation. Attackers use iterative or adversarial queries to leverage model outputs to infer the structure, content, or source of the retrieved context. Successful attacks can compromise intellectual property, achieve data laundering, or facilitate circumvention of access controls on restricted material.

[0027] The following will explain how to defend against this attack based on an exemplary question-and-answer RAG system.

[0028] Generally, the operation of RAG systems, especially question-answering RAG systems, involves embedding text, prompts, and queries into a latent space. Lexicalization and embedding are handled here by... This function represents the text. Mapping to its embedding ,in It is the length of the token, and It refers to the embedding size.

[0029] set up This is a system prompt template, designed to prompt a user when they query... and the retrieved context or The retrieved context ( In the case of integers, guide the LLM toward the desired answer. . , , The embeddings are respectively represented as , and The retrieved context is obtained as follows: ,in It is based on embedding similarity or BM25 permutation, from a larger set ( It is an integer, where The retriever function selects the relevant context.

[0030] By weight Parameterized LLM in a given , and Generate an answer under the following circumstances The process is represented as: For the sake of brevity, and They are omitted because they remain fixed.

[0031] In this setup, the attacker's objective is to send a carefully crafted attack query to the RAG system 100. To extract the context set Or a semantically equivalent version. Attackers aim to maximize attack success metrics, such as the success rate of a given attack query. In this case, maximize the answer The n-grammatical overlap or semantic similarity between the retrieved context and the retrieved context. The attacker operates without access to the following: system prompt templates. , context set LLM functions and parameters Embedded functions Retrieval function The RAG system 100 was designed to handle related queries. and the desired response. .

[0032] The defender's goal is to implement a strategy that avoids RAG context extraction while preserving the utility of the context. After applying the strategy, a query is performed. The protected answer is represented as Therefore, the objective is: (i) utility preservation: ensuring In task-specific test datasets ( Reserved on integers The expected function, and (ii) extraction prevention: ensure that in the response The extracted context deviates significantly from the actual retrieved context. The defender can access the model functions and parameters. Lexicalization and embedding functions Retrieval function and system prompt templates However, they cannot distinguish between malicious and benign queries and lack prior knowledge of attacker strategies.

[0033] According to this disclosure, a method for context extraction defense in RAG-enhanced LLMs is provided, which replaces the original context with a proxy context that preserves the functionality of the original context.

[0034] Figure 2 This is a flowchart of an exemplary method 200 for defending against context extraction in a large language model enhanced through retrieval enhancement. Method 200 includes: - Calculate a 210-optimized proxy context embedding for the original context embedding, which is: - Minimize the difference between the responses to regular queries generated by the large language model based on the original context embedding and the responses to regular queries generated by the large language model based on the optimized proxy context embedding, and - Maximize the difference between the response to the attack query generated by the large language model based on optimized proxy context embedding and the original context; and - Based on optimized proxy context embedding, 220 pairs of responses to input queries are generated by a large language model.

[0035] The above method can be applied to the original context set. Each context in .

[0036] The defense methods will be explained in detail below, based on the original context. and their embedding The defender wants to obtain new context embeddings. The new context embedding (1) Minimize the original context under normal operating conditions The response difference between the context proxy and the attack query, and simultaneously (2) maximizes the attack query. The model's answer and context Dissimilarity between them. To this end, the defender sets the expected use of the context. One query The set ( (It is an integer). Given the context, such a query can be synthesized using LLM. Using this notation, the first objective can be written as: , in It's cross-entropy loss. The difference between the original suggestion and the model's response to the attacker's query can be minimized by reducing the negative cross-entropy loss between these values. And thus maximized. The defender's two objectives can be combined into the following optimization problem: .

[0037] Defenders, due to their lack of attack queries Therefore, the above optimization problem cannot be solved directly due to the limited access. Instead, they can use a fixed query. As an attack query The agent. It might be a trivial attack strategy, and perhaps not even intended for a successful attack; instead, it might simply be used by the defender in the optimization process, and could serve as a lower bound for a potential attacker's query. The optimization problem above can be rewritten as: .

[0038] Minimize the given The answer and the original context Negative cross-entropy loss at the lexical level may not guarantee semantic dissimilarity. To meet this requirement, the answer and fixed target context are needed. The losses between these losses, specified by the defender, are semantically distinct; for example, "Stop stealing context!" can be minimized instead. The ultimate joint objective (i.e., the ultimate optimization problem) can be defined as follows: .

[0039] The final optimization problem defined above can be solved by the defender based on the information available to them. However, optimizing each context directly in large datasets is not feasible. of Computationally, this can be complex, as even a single optimization step can take minutes to perform fine-grained control. To scale contextual agents to large amounts of data, it is possible to use agent generator neural networks. The proxy generator neural network Learning to approximate any input context Optimized proxy embedding .

[0040] Proxy Generator Neural Network It can be trained to be a differentiable approximation of the original optimization, inheriting its safety guarantees, while achieving low latency per context.

[0041] Proxy Generator Neural Network Able to base on a given original context dataset The generated training dataset And trained, and includes pairs of original context embeddings. and the corresponding optimized surrogate context embedding calculated by solving the final optimization problem described above. .here, It is an integer representing the total number of pairs in the dataset. (Proxy generator neural network) Learning to embed the original context Mapping to the corresponding proxy context embedding Proxy generator neural network Trained by solving the following: , in Represents a proxy generator neural network The parameters, Represents the original context dataset, and Represents the original context dataset The original context.

[0042] Once trained Optimized proxy context embeddings are generated in a single forward pass, eliminating the need for per-context optimization. For critical or outlier contexts where the proxy generator's predictions perform poorly (e.g., low-confidence embeddings detected via outlier scores), a lightweight fine-tuning step can refine the proxy context embeddings using the final optimization problem described above, thus ensuring robustness without sacrificing scalability.

[0043] The optimization of such a proxy generator model can also be combined with the final optimization problem above, for example as follows: .

[0044] here, Indicates including A pair of contexts and their corresponding queries The dataset, in which It is an integer.

[0045] The method according to this disclosure can be employed in a diagnostic RAG system. Such a system can be configured to make a medical diagnosis in response to a query that includes patient symptoms. In such a system, a user can enter a query such as: “Whenever I move my head or stand up quickly, the room starts spinning violently. I feel nauseous but cannot vomit, and lying down still relieves the symptoms.” In response to such a query, contextual data associated with private patient data (such as name and age) can be retrieved from a patient database storing symptoms and corresponding diagnoses. The relevant context retrieved in response to the exemplary symptoms above could be: “Patient (John Doe, 45 years old): The room spins when I move; Doctor: Possible cause: Benign paroxysmal positional vertigo (BPPV).”

[0046] For data protection reasons, it is extremely important to protect private patient data (such as name and age) from unauthorized access and to forward only diagnostic data to the LLM when responding to requests. This can be achieved by generating an optimized proxy context embedding according to the method of this disclosure, and then combining this optimized proxy context embedding with the query and system prompt (e.g., "Please answer the query based on the retrieved context") into a coherent prompt that is forwarded to the LLM, which has the task of formulating a response. The LLM might output a response like, "You may be experiencing benign paroxysmal positional vertigo (BPPV)," which is similar to the diagnostic data stored in the database but hides the private patient data stored in the database.

[0047] The method according to this disclosure can be used by Figure 3 The data processing apparatus 300, schematically shown in the diagram, is implemented. The data processing apparatus 300 may include a processing device 310 and a computer-readable storage medium 320 connected to the processing device 310. The processing device 310 may include or be configured as one or more general-purpose processing devices (such as a processor), or be configured as one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and the like. The computer-readable storage medium 320 may include or be configured as read-only memory (ROM), random access memory (RAM), and the like.

[0048] A computer program may be stored in a computer-readable storage medium 320, which includes instructions executable by a processing device 310 (i.e., a computer). The instructions included in the computer program cause the processing device 310 to perform the methods described above.

[0049] The data processing device 300 may include an input / output unit 330 configured to receive and process queries input by the user 340. And the processed query The query is forwarded to the processing device 310. Upon receiving the query, the processing device 310 can retrieve the relevant original context from the context database 360 ​​that stores the context set.

[0050] Context can be embedded as the original context. The original context embedding is stored in database 360. In this case, processing device 310 can be configured to retrieve the relevant original context embedding from database 360. And compute the optimized context embedding as described above. .

[0051] Alternatively, Database360 can retrieve the original context. Stored as raw data. In this case, the processing device 310 can be configured to first use the embedding function. The original data is embedded into the latent space, and then the optimized context embedding is computed as described above. .

[0052] Processing device 310 can then query based on input. and optimized context embedding Hints for coherent composition and the prompt The input is fed into the large language model 370. The large language model 370 can be implemented by the processing device 310 by executing instructions stored in the storage medium 320. The large language model 370 can be configured based on optimized agent context embedding. To generate a query against the input The protected answer Then, the response to protection. It can be output to user 340 via input / output unit 330.

Claims

1. A computer-implemented method (200) for defending against context extraction in a large language model (370) enhanced by retrieval enhancement, the method (200) comprising: - Embedded in the original context ( ) compute (210) optimized agent context embedding ( The optimized proxy context embedding ( ): - Minimize the embeddings of the large language model (370) based on the original context ( The generated response to a regular query and the optimized proxy context embedding by the large language model (370) The differences between the generated responses to the regular query, and - Maximize the agent context embedding based on the optimized proxy context by the large language model (370). The differences between the generated response to the attack query and the original context, and - Based on the optimized proxy context embedding ( The large language model (370) generates (220) a response to the input query.

2. The method (200) according to claim 1, wherein the optimized proxy context embedding ( The calculation of (210) is performed by the following: - Minimize the original context embeddings based on the large language model (370) The response generated by the large language model (370) to the predetermined query set expected to be used in the original context and the response generated by the large language model (370) based on the proxy context embedding ( The cross-entropy loss between the generated responses to the predetermined query set, and - Minimize the proxy context embeddings of the large language model (370) The cross-entropy loss between the generated response to a fixed query and the fixed target context.

3. The method (200) according to claim 2, wherein the optimized proxy context embedding ( ) is calculated by solving the following optimization problem: ， in Indicates the original context, This refers to the pre-defined query set. Represents query set The query, Indicates query Embedded, Indicates the original context Embedded, This indicates the embedding of a fixed query. This indicates that the proxy context is embedded. This indicates a fixed target context, and This represents the cross-entropy loss.

4. The method (200) according to claim 2 or 3, wherein the queries in the query set are synthesized by means of the large language model (370) using the original context as a reference.

5. The method (200) according to claim 3 or claims 3 and 4, wherein the optimized proxy context embedding ( The computation of is approximated by a proxy generator neural network, which is trained by a training method comprising: - Provides a primitive context dataset that includes multiple primitive contexts; - Calculate the corresponding original context embedding for each original context in the context dataset; - By solving the optimization problem defined in claim 3, the corresponding proxy context embedding is calculated for each original context embedding; - Provides training data including pairs of original context embeddings and corresponding proxy context embeddings; - Train the agent generator neural network to learn to map the original context embedding to the optimized agent context embedding.

6. The method (200) of claim 5, wherein training is performed by solving the following: ， in This represents the parameters of the proxy generator. This refers to the original context dataset. Represents the original context dataset The original context, This represents the agent generator neural network. This indicates the original context embedding, and This indicates that the proxy context is embedded.

7. The method (200) of claim 5, wherein training is performed by solving the following: ， in This represents a dataset that includes paired contexts and the queries corresponding to those contexts. Indicates the original context, Indicates a query. Indicates query Embedded, This indicates the embedding of the original context. This indicates the embedding of the fixed query. This represents a mapping of the original context embedded by the neural network of the proxy generator. This indicates a fixed target context, and This represents the cross-entropy loss.

8. A data processing apparatus (300) comprising a component (310) configured to perform the method according to any one of claims 1 to 7.

9. A computer program comprising instructions which, when executed by a computer (310), cause the computer (310) to perform the method (200) according to any one of claims 1 to 7.

10. A computer-readable storage medium (320) including instructions which, when executed by a computer (310), cause the computer (310) to perform the method (200) according to any one of claims 1 to 7.

Images