Method for defending context extraction in retrieval-based context learning enhanced large language model

Abstract

A computer-implemented method for defending against example extraction in a large language model enhanced with retrieval-based contextual learning includes computing optimized proxy example embeddings for original example embeddings of examples stored in an example database by minimizing a cross-entropy loss between responses generated by the large language model based on original example embeddings for a predetermined set of queries and responses generated by the large language model based on proxy example embeddings for the predetermined set of queries and minimizing a cross-entropy loss between responses generated by the large language model based on proxy example embeddings for an attack query and a fixed predetermined response; and generating, by the large language model, a response to an input query based on the optimized proxy example embeddings.

Description

Technical Field

[0001] This disclosure relates to a method for defending against context extraction in a large language model enhanced by retrieval-based context learning. Background Technology

[0002] Ret-Based Context Learning (RetICL) is a method to enhance Large Language Models (LLMs) by dynamically integrating relevant examples retrieved from external data stores. During interaction, the RetICL system integrates two basic components: user queries and retrieved examples, which serve as demonstrations to enrich contextual understanding and improve response accuracy. Summary of the Invention

[0003] This disclosure addresses scenarios where retrieved examples should be protected and inaccessible to end users of the RetICL system. This is particularly critical when such systems are deployed as external services by private or corporate entities. RetICL providers may use proprietary, privacy-sensitive, or confidential data to improve model performance. Unauthorized access to these retrieved examples could allow attackers to reconstruct the underlying dataset, thereby exposing valuable intellectual property or confidential domain knowledge.

[0004] Therefore, a resilient defense mechanism is needed to protect the confidentiality of training data while maintaining the effectiveness of the response.

[0005] According to a first aspect of this disclosure, this objective is achieved by a computer-implemented method for defending example extraction in a large language model enhanced by retrieval-based context learning, the method comprising: The optimized proxy instance embedding is computed against the original instance embedding of, for example, an instance stored in an instance database, in the following manner: - Minimize the cross-entropy loss between the responses generated by the large language model based on the original example embeddings and the responses generated by the large language model based on the proxy example embeddings for the predetermined query set, and - Minimize the cross-entropy loss between the responses generated by the attack query and the fixed, predetermined responses based on the proxy example embeddings in the large language model; and - The response to the input query is generated by a large language model based on optimized proxy example embeddings.

[0006] The utility of Retrieval-Based Context Learning (RetICL)-enhanced Large Language Models (LLMs) is preserved by minimizing the cross-entropy loss between the responses generated by the large language model based on the original example embeddings for a predetermined query set and the responses generated by the large language model based on the proxy example embeddings for the predetermined query set. Furthermore, minimizing the cross-entropy loss between the responses generated by the large language model based on the proxy example embeddings for attack queries and fixed predetermined responses prevents the extraction of original examples.

[0007] The queries in this set of pre-defined queries can be synthesized using the original example as a reference within a large language model. As attack queries, proxy queries can be used. For example, it might only be used during optimization and can serve as a lower bound for potential attack queries. Fixed pre-defined responses can be specified to be semantically different from the original example.

[0008] The optimized proxy example embedding can be computed by solving the following optimization problem: , in Let q represent the pre-defined query set. The query in This indicates the embedding of query q. This represents the embedding of one or more original examples retrieved in response to an input query q. This indicates that a proxy example is embedded. This indicates an attack query embedding. This indicates a fixed, predetermined response, while This represents the cross-entropy loss.

[0009] The optimized proxy instance embedding obtained by solving the above optimization problem can be used as a single global proxy instance embedding, which can be combined with the corresponding embedding for each query.

[0010] To effectively handle complex or heterogeneous patterns, multiple proxy instance embeddings may be required. To address this, the method may further include: learning a set of M proxy instance embeddings. and coefficient matrix Where N is the number of instances stored in the database or the total number of instances stored in the database. And W im ≥0. For example, learning can be achieved by minimizing the above expression, where the proxy example embedding... Replaced with a convex combination of M proxy example embeddings, where the coefficient matrix W im The coefficient is used as the coefficient.

[0011] Therefore, the proxy instance embeddings for each example stored in the dataset can be represented in terms of the learned M proxy instance embeddings as follows: , in This represents the proxy instance embedding of the i-th instance among N instances. Represents the collection of learned proxy example embeddings An embedded example of a learned proxy, while This represents the coefficients of the learned coefficient matrix.

[0012] If k relevant examples are retrieved in response to an input query, the proxy example embedding can be determined by concatenating the corresponding proxy example embeddings for these k examples as follows: , in This indicates the final proxy example embedding. This represents the proxy instance embeddings of the retrieved k instances, while the symbol... This indicates a join operation. In this way, rich structural information can be preserved.

[0013] Alternatively, the proxy instance can be determined by aggregating the proxy instance embeddings of the retrieved k instances in the following manner: .

[0014] In this case, the collection of proxy instance embeddings can be optimized by addressing the following issues. sum coefficient matrix Learning: .

[0015] In a second aspect of this disclosure, a data processing apparatus is provided, which includes means configured to perform the methods described above.

[0016] In a third aspect of this disclosure, a computer program is provided that includes instructions that, when executed by a computer, cause the computer to perform the method as described above.

[0017] In a fourth aspect of this disclosure, a computer-readable storage medium is provided, comprising instructions that, when executed by a computer, cause the computer to perform the method as described above. Attached Figure Description

[0018] This disclosure will be described in detail below with reference to the accompanying drawings. In the drawings: Figure 1This is a schematic diagram showing an example of a RetICL-enhanced LLM.

[0019] Figure 2 This is a flowchart of an exemplary method according to this disclosure.

[0020] Figure 3 This is a schematic diagram of an exemplary data processing apparatus according to the present disclosure. Detailed Implementation

[0021] Figure 1 is a schematic diagram showing an example of a large language model (LLM) enhanced by retrieval-based context learning (RetICL), hereinafter referred to as RetICL system 100.

[0022] Retrieval-based contextual learning is a method to enhance large language models by dynamically integrating relevant examples retrieved from external data stores. Figure 1 illustrates an example of the workflow of the RetICL system 100. In response to a user's input query 110 in the retrieval unit 120, the retrieval unit 120 selects relevant examples from training data stored in a database 130 based on a predefined similarity metric. These examples are combined with the input query 110 to synthesize a prompt 140, which is then used as a contextual demonstration of the large language model 150, whose task is to formulate a response 160. The LLM 150 utilizes these retrieved examples to accurately interpret and respond to user input within context, significantly improving the quality and accuracy of its output.

[0023] A user's query might be, for example, a review like "The food is delicious." In response to this review, the task of the LLM 150 might be to formulate a response that reflects the opinion associated with the input review. Retrieved examples might include multiple reviews such as "Very delicious!", "Very bad!", "Delicious!", and "I like it!", along with corresponding opinions such as "Positive", "Negative", "Positive", and "Positive". These examples are combined with the input review "The food is delicious!", and the task of the LLM 150 is to formulate a response that reflects whether the review is "positive" or "negative".

[0024] RetICL extraction attacks target RetICL-enabled systems to infer, reconstruct, or extract proprietary training examples used for retrieval augmentations. Attackers can design iterative or adversarial queries to systematically reveal retrieved examples, which may contain sensitive data, proprietary datasets, or curated domain knowledge. Successful attacks compromise intellectual property, lead to data breaches, and facilitate unauthorized copying of specialized training data.

[0025] The following explanation will use the exemplary RetICL system to illustrate methods for defending against such attacks.

[0026] Typically, operations in the RetICL system involve embedding text, hints, and queries into a latent space. The tokenization and embedding functions are denoted here as... This function maps text X to its embedding. , where n X Let be the tag length, and e be the embedding size. For a given user query q, the retrieval function R(q) is derived from the training dataset based on a similarity metric. Return the first k relevant examples (k is an integer), where N is the dataset size. Each example e is a question-answer pair (q, a). Then, by Parameterized LLM based on Generate a response For the sake of simplicity, the parameters have been omitted. Because they are fixed. and These are the retrieved examples and the embedded query, respectively.

[0027] In this setup, the attacker's goal is to submit a carefully crafted attack query to the RetICL system. To extract the dataset The attacker aims to maximize metrics such as model output or semantically equivalent information. Embedding similarity or n-gram overlap between the original training examples and the actual training dataset. Attackers without access to the complete training dataset... LLM internal parameters LLM functions and embedded functions This can be done under normal circumstances; however, attackers can query the system and observe its output.

[0028] The defender's goal is to ensure that the required functionality is preserved in response to user queries, while preventing any attacker from reconstructing the training examples. This assumes the defender has access to the complete training dataset. LLM functions Embedded functions and retrieval functions However, defenders cannot distinguish between benign and malicious queries.

[0029] According to this disclosure, a method for defending against instance extraction in a RetICL-enhanced LLM is provided, which replaces the retrieved instance (or instance embedding) with a proxy instance (or proxy instance embedding) to preserve utility while preventing extraction attacks.

[0030] Figure 2 This is a flowchart of an exemplary computer implementation of a method 200 for defending example extraction in a large language model enhanced by retrieval-based context learning, wherein the method 200 includes: - The optimized proxy instance embedding is computed for example, for instance, instances stored in an instance database, using the following method: - Minimize the cross-entropy loss between the responses generated by the large language model based on the original example embeddings and the responses generated by the large language model based on the proxy example embeddings for the predetermined query set, and - Minimize the cross-entropy loss between the response generated by the attack query and the fixed, predetermined response based on the proxy example embedding in the large language model; and - 220 responses to the input query are generated by the large language model 370 based on optimized proxy example embeddings.

[0031] This defense method will be explained in detail below in the RetICL setup, where the large language model is improved by using the training dataset. The example set R(q) is retrieved to answer the user query q. In the standard framework of existing technology, the retrieved examples are directly used by the LLM, which increases the risk of the training data being extracted. To mitigate this risk, the original retrieved examples are replaced by a proxy that retains the intended utility while preventing extraction.

[0032] The retrieved instance (instance embedding) may be replaced by a global proxy (global proxy embedding), meaning the defender seeks to learn a single proxy instance embedding. When embedded with a query When combined, the proxy example embedding approximates the output of a retrieval-based system (i.e., a RetICL-enhanced LLM). The defense objective is twofold: (i) to preserve utility, minimize the LLM using the originally retrieved example. Output and use of agents The differences between the outputs; and (ii) to prevent extraction, in adversarial attack queries Below, force the output Deviating from any direct reconstruction of the training examples, which is relative to the predefined cue. It is measured using cross-entropy as the loss function. This goal can be formalized as: .

[0033] Here, q represents the set of questions / queries in the training data, since the defender may not have access to user queries. One of the problems, It is a query embedding. It is the embedding of the response to query q (i.e., the embedding of one or more retrieved examples). It is proxy embedding, and This indicates a predefined prompt, i.e. a fixed, predetermined response.

[0034] A single global proxy may not be sufficient because queries can be too complex or contain heterogeneous patterns that cannot be effectively captured by a single instance as context. To overcome this limitation, a set of M instance proxies can be learned. and a corresponding coefficient matrix The method is extended to include integers N and M. In this way, the dataset... Each instance in the list can be a convex combination of M instance proxies. Replaced, of which W im Representative Agent In approximation The weight of time, where .

[0035] In this way, for each query q, the proxies of the k retrieved examples can be aggregated to form a new retrieved representation. As Agent: .

[0036] Alternatively, the retrieved proxies can be joined to preserve richer structural information: , Each of them Represents the proxy embedding for the i-th retrieved example, and the symbol This indicates a join operation.

[0037] The current defense objective is to ensure the use of a proxy. The output closely matches the output obtained from the original retrieved examples, thus preserving the utility of the retrieval-based system. Simultaneously, by requiring [the system to perform certain actions] for attack queries... The output of each individual example agent is directed toward the pre-defined response. To enforce the prevention of extraction. Formally, if the proxy is aggregated, the optimization objective is: .

[0038] In this extended formula, using M shared example agents ensures that each agent has access to a wide range of training examples, thereby learning robust and generalizable representations. The coefficient matrix W further ensures that the local features of each retrieved example are captured. This approach achieves a balance between preserving utility and preventing extraction in the RetICL setting.

[0039] The method according to this disclosure can be implemented by the data processing apparatus 300 schematically shown in FIG3. The data processing apparatus 300 may include a processing device 310 and a computer-readable storage medium 320 connected to the processing device 310. The processing device 310 may include or be configured as one or more general-purpose processing devices, such as a processor, or one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. The computer-readable storage medium 320 may include or be configured as a read-only memory (ROM), random access memory (RAM), etc.

[0040] A computer program may be stored in a computer-readable storage medium 320, including instructions executable by a processing device 310, i.e., a computer. The instructions contained in the computer program may cause the processing device 310 to perform the methods described above.

[0041] The data processing apparatus 300 may include an input / output unit 330 configured to receive and process a query q input by a user 340, and forward the processed query q to the processing device 310. Upon receiving the query, the processing device 310 may retrieve relevant original examples from a sample database 360 ​​storing sample sets.

[0042] These examples can be embedded as original examples. The data is stored in database 360. In this case, as described above, processing device 310 can be configured to retrieve the relevant original example embedding from database 360. And compute the optimized example embedding. .

[0043] Alternatively, database 360 ​​can store the original example as raw data. In this case, as described above, processing device 310 can be configured to first use the embedding function. To perform embeddings from the raw data into the latent space, and then compute optimized example embeddings. .

[0044] Then, the processing device 310 can, based on the input query q and the optimized example embedding A coherent cue P is synthesized and input into the large language model 370. The large language model 370 can be implemented by the processing device 310 by executing instructions stored in the storage medium 320. The large language model 370 can be configured based on optimized proxy example embeddings. Generate a secure answer to the input query q. Safe response It can then be output to user 340 through input / output unit 330.

Claims

1. A computer-implemented method (200) for defending against example extraction in a large language model (370) enhanced by retrieval-based context learning, the method (200) comprising: - Embed the original example in the following way ( ) to compute (210) optimized proxy example embedding ( ): - Minimize the response generated by the large language model (370) based on the original example embeddings to the predetermined query set and the response generated by the large language model (370) based on the proxy example embeddings. The cross-entropy loss between the responses generated for the predetermined query set, and - Minimize the proxy example embedding in the large language model (370) The cross-entropy loss between the response generated by the attack query and the fixed, predetermined response; and - The large language model (370) is based on the optimized proxy example embedding ( Generate (220) response to the input query.

2. The method (200) according to claim 1, wherein, The optimized proxy example embedding ( This is calculated by solving the following optimization problem: , in Let q represent the pre-defined query set. The query in This indicates the embedding of query q. Indicates the embedding of the original example. This indicates that a proxy example is embedded. This indicates an attack query embedding. This indicates a fixed, predetermined response, while This represents the cross-entropy loss.

3. The method (200) according to claim 1 or 2, wherein the method comprises: Learn a collection of M proxy example embeddings and coefficient matrix Where N is the number of examples stored in the database. And W im ≥0.

4. The method (200) according to claim 3, wherein, Optimized proxy instance embedding of the i-th example among N examples Determined in the following manner: ， in Represents the collection of learned proxy example embeddings One of the learned proxy example embeddings, while W im This represents the coefficients of the learned coefficient matrix.

5. The method according to claim 4, further comprising: For the k instances retrieved in response to the input query q, the proxy instance embedding is determined by concatenating the corresponding proxy instance embeddings of the k instances as follows: , in This indicates the final proxy example embedding. This represents the proxy instance embeddings of the retrieved k instances, while the symbol... This indicates a join operation.

6. The method of claim 4, further comprising determining a proxy instance embedding for k instances retrieved in response to an input query q by aggregating the proxy instance embeddings of the k retrieved instances in the following manner: 。 7. The method according to claim 6, wherein, The collection of proxy instance embeddings is achieved by addressing the following optimization issues. sum coefficient matrix Learning: 。 8. A data processing apparatus (300) comprising means (310) configured to perform the method according to any one of claims 1 to 7.

9. A computer program comprising instructions that, when executed by a computer (310), cause the computer (310) to perform the method (200) according to any one of claims 1 to 7.

10. A computer-readable storage medium (320) comprising instructions that, when executed by a computer (310), cause the computer (310) to perform the method (200) according to any one of claims 1 to 7.

Images