Multi-stage interpretable adversarial defense method and system for retrieval-augmented generation system

Abstract

This invention relates to a multi-stage interpretable adversarial defense method and system for retrieval enhancement generation systems. The method includes: constructing a candidate document set; introducing a dual filtering strategy to perform anomaly detection and pre-filtering on the candidate document set to obtain an intermediate document set; prompting an LLM (Limited Language Module) to generate a baseline answer based on its internally stored knowledge without introducing external documents; using the LLM to compare each intermediate document in the intermediate document set with the baseline answer, generating a structured explicit reasoning result for the corresponding intermediate document, and constructing a target document set; comprehensively evaluating the confidence level of the intermediate document set based on the structured explicit reasoning results of all intermediate documents; selecting an adaptive fusion mode based on the confidence level, and using the LLM to provide the answer under this adaptive fusion mode; the system is used to implement the above method. Compared with the prior art, this invention optimizes the shortcomings of existing RAG (Retrieval Enhancement Generation) system defense schemes in terms of decision transparency, fusion flexibility, and the balance between efficiency and security.

Description

Technical Field

[0001] This invention relates to the field of adversarial detection technology, and in particular to a multi-stage interpretable adversarial defense method and system for retrieval enhancement generation systems. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in question-answering and dialogue systems, Retrieval-Augmented Generation (RAG) has become a key technological paradigm for improving generation quality and factual accuracy. RAG systems significantly improve the accuracy, timeliness, and credibility of answers by combining the parameterized internal knowledge of the LLM with massive amounts of external non-parameterized knowledge, through retrieving relevant documents from external knowledge bases before generating answers. However, due to this open reliance on external knowledge, RAG systems face unprecedented security threats, among which Corpus Poisoning Attacks are one of the most direct and insidious. Attackers do not need to tamper with model parameters; they only need to inject a small number of carefully crafted malicious documents into the open knowledge bases (such as Wikipedia, forums, and code repositories) upon which the RAG system relies. These adversarial documents are semantically optimized and can be recalled with high priority by the retrieval system when querying specific users, thus polluting the LLM context and inducing the model to generate pre-set incorrect or even harmful answers. Such poisoning attacks take many forms, including HotFlip white-box attacks based on gradient-optimized text, black-box attacks using language models to generate semantically fluent adversarial paragraphs, and Prompt Injection Attacks (PIA) that embed hidden instructions into documents. These attacks can significantly weaken the reliability of RAG systems, leading to serious security consequences.

[0003] To address these threats, the academic community has proposed a series of defense mechanisms, including two main categories: post-retrieval detection filtering and response generation process optimization. Among these, post-retrieval detection filtering aims to identify and remove malicious documents after RAG retrieval but before generation, thus preventing poisoned information from entering the LLM generation stage. For example, Chinese patent application CN119829705A provides a knowledge question-answering adversarial detection method for retrieval-enhanced large language models. This method uses explicit prompts and contextual semantic substitution and unsupervised adversarial detection to resist adversarial attacks targeting both the question and the local knowledge base.

[0004] While existing defense methods have achieved some success in specific scenarios, overall, the security protection of the RAG system still has the following defects and shortcomings: 1) Existing solutions usually only output the fused answer without clearly indicating which documents are considered suspicious and the reasons for the judgment. This lack of interpretability in the defense process seriously affects the system's credibility, auditability, and debuggability; 2) Most current hybrid defense frameworks adopt fixed knowledge fusion prompting processes and cannot dynamically adjust strategies according to threats; 3) Efficiency and robustness are difficult to balance. There is a lack of a layered defense mechanism that can ensure a high detection rate while keeping expensive LLM calls within a necessary range, thereby balancing system performance and defense strength.

[0005] Therefore, providing a method to optimize the shortcomings of existing RAG system defense schemes in terms of decision-making transparency, integration flexibility, and the balance between efficiency and security is a technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a solution.

[0007] The objective of this invention can be achieved through the following technical solutions: According to a first aspect of the present invention, a multi-stage interpretable adversarial defense method for retrieval enhancement generation systems is provided, comprising: The pre-built knowledge base is queried and retrieved to construct a candidate document set. A dual filtering strategy is introduced to perform anomaly detection and pre-filtering on the candidate document set, and candidate documents with obvious malicious intent are removed to obtain an intermediate document set. The dual filtering strategy is based on the results of combining semantic embedding clustering detection and text N-gram similarity detection to filter documents. Without incorporating external documentation, the LLM is prompted to generate benchmark answers based on the knowledge stored internally. LLM is used to compare each intermediate document in the intermediate document set with the benchmark answer to generate a structured display reasoning result for the corresponding intermediate document. The intermediate document set is then filtered based on the structured display reasoning result to obtain the target document set. The structured display reasoning result includes document number, judgment tag, and reason key-value pairs. Based on the structured display inference results of all intermediate documents, the confidence level of the intermediate document set is comprehensively evaluated. Based on the aforementioned confidence level, an adaptive fusion mode is selected. In this adaptive fusion mode, the user query and the target document set are input into the LLM to generate an answer. The adaptive fusion mode includes a high-security mode and a normal mode.

[0008] As a preferred technical solution, the candidate document set is constructed from the Top-K documents with the highest relevance in the knowledge base.

[0009] As a preferred technical solution, the method combining semantic embedding clustering detection is as follows: Each candidate document in the candidate document set is mapped to a fixed-dimensional vector representation using a semantic embedding model; The vector representation is clustered using a density-based unsupervised clustering algorithm, and the candidate documents for poisoning attacks and abnormal candidate documents are detected based on the clustering analysis results. The candidate documents to be poisoned are: in the clustering results, the candidate documents corresponding to the clusters whose density differs from that of the surrounding clusters by more than a first density threshold; The abnormal candidate documents are: candidate documents corresponding to isolated cluster points in the clustering results.

[0010] As a preferred technical solution, the method for text N-gram similarity detection is as follows: The candidate documents in the aforementioned candidate document set are randomly grouped into pairs, and the following operations are performed on each pair of candidate documents: Calculate the overlap of the longest common subsequence between each pair of candidate documents. If the overlap is greater than a first overlap threshold, mark the corresponding candidate document pair as a suspicious candidate document.

[0011] As a preferred technical solution, the dual filtering strategy is as follows: Candidate documents that are not subject to poisoning attacks, are not abnormal, or are not suspicious are retained. Retain the candidate documents for poisoning attacks corresponding to clusters whose density difference with the density of surrounding clusters is greater than the first density threshold but less than the second density threshold, and remove the remaining candidate documents for poisoning attacks. Retain suspicious candidate documents whose overlap is less than the first overlap threshold but greater than the second overlap threshold, and remove the remaining suspicious candidate documents.

[0012] As a preferred technical solution, the method for generating the structured display reasoning result is as follows: setting the LLM prompt includes: determining whether the intermediate document conflicts with the baseline answer, the type of conflict, and the severity; inputting each intermediate document in the intermediate document set into the LLM, asking questions according to the prompt, and outputting the structured display reasoning result, including document ID, judgment label, and reason key-value pairs.

[0013] As a preferred technical solution, the method for obtaining the confidence level is as follows: based on the structured display reasoning results, obtain confidence level features, including the number and proportion of intermediate documents with conflict labels, the distribution of conflict severity, and the repetition rate of the reason key-value pairs; and calculate a confidence score in the range of 0 to 1 based on the confidence level.

[0014] As a preferred technical solution, obtaining the confidence level further includes detecting attack feature phrases for each intermediate document; if an attack feature phrase is detected, the confidence level of the corresponding intermediate document is directly set to zero.

[0015] As a preferred technical solution, when the confidence level is less than or equal to the confidence threshold, the adaptive fusion mode is a high-security mode, in which the LLM is limited to using its internal knowledge to answer; when the confidence level is greater than the confidence threshold, the LLM's internal knowledge is combined with the target document set to answer according to the standard RAG process.

[0016] According to a second aspect of the present invention, a multi-stage interpretable adversarial defense system for retrieval enhancement generation systems is provided for implementing the above-described method.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1) Employing a multi-stage collaborative defense architecture improves the robustness of attack detection. This invention breaks down the defense process into three complementary stages. First, unsupervised clustering and similarity comparison are used to filter out most abnormal documents. Second, LLM deep analysis is used to analyze the knowledge conflicts between the remaining documents and the basic answers, and the confidence of the document set is calculated. Finally, the answer generation strategy is dynamically adjusted based on the confidence. This layered defense framework compensates for the limitations of single defense methods, significantly enhances the system's ability to resist diverse attacks, and has stronger defense robustness than existing methods. It can identify and isolate poisoned documents and attack methods through multi-layered means, regardless of the number of poisoned documents or the attack method.

[0018] 2) The introduction of lightweight pre-filtering and on-demand inference mechanisms significantly improves system efficiency. Compared to single-stage defense schemes that require frequent calls to large models for judgment, most suspicious documents in this invention are filtered out in the first stage, thereby reducing the number of documents entering LLM inference. This allows LLM to perform conflict analysis only on a small number of edge cases, greatly reducing average computational cost and response latency. By limiting the most complex LLM inference to the necessary range and using simple and fast detection algorithms for pre-screening, the system's throughput and latency during attack defense remain on the same order of magnitude as without defense. Therefore, this invention significantly improves security while having minimal impact on the system performance of practical applications, meeting the real-time requirements of high-concurrency scenarios.

[0019] 3) The introduction of an explicit reasoning chain in defense decisions significantly improves the interpretability of the process. Traditional methods such as TrustRAG and AstuteRAG rely solely on internal LLM assessments to provide the final answer, constituting a black-box decision-making process. Users cannot understand why the system trusts or suspects certain documents. This invention requires the LLM to output a structured judgment process, clearly indicating whether each document is conflicting and the reasons, transforming defense from a "black box" to a "white box." This improvement makes the internal judgment criteria of the defense system transparent and visible to users and administrators, significantly enhancing the system's verifiability and auditability. In practical applications, security auditors can trace the source of judgments based on the reasoning chain provided by the LLM, quickly understanding which documents have been poisoned and how they have been tampered with, thereby more effectively locating risks and optimizing the system. Attached Figure Description

[0020] Figure 1 This is a technical framework diagram of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] Existing technologies have proposed a series of defense mechanisms against threats emerging in language models. Early heuristic detection methods, such as using text perplexity (PPL) to screen for anomalous documents, have found that well-designed adversarial documents often highly overlap with normal documents in terms of PPL distribution, leading to insufficient robustness and a high risk of misjudgment. Recent work has shifted towards using inter-document relationships for detection. For example, the TrustRAG method introduces clustering ideas, assuming that malicious documents generated from similar templates will form high-density clusters in the embedding space, and thus uses K-means clustering to identify anomalous clusters. Another example is the RevPRAG method, which uses internal activation features from the LLM generation stage to detect attacks. It finds that poisoned answers differ significantly from normal answers in cross-layer activation distribution, thus identifying poisoned outputs by comparing activation vectors. Defense methods based on process optimization improve robustness by modifying the answer generation process of the RAG. For example, RobustRAG employs an "isolation-re-aggregation" strategy: the LLM generates independent answers for each retrieved document, and then these answers are merged through voting. This strategy is effective when malicious documents are in the minority, but when malicious documents constitute the majority of the search results, the simple voting mechanism fails, and attacks can still succeed. More advanced methods leverage the powerful internal knowledge of the LLM for conflict detection: InstructRAG fine-tunes the model to learn to explicitly identify and filter out noisy information; AstuteRAG guides the LLM to first generate a baseline answer based on its internal knowledge, then compares it with external retrieved documents to discover knowledge conflicts and remove malicious content, finally generating the answer. These methods utilize the LLM's own judgment to improve the success rate of defense to some extent.

[0023] However, the above methods still have the following problems: 1) Lack of interpretability in the defense decision-making process. Many defense methods that rely on LLM for conflict arbitration (such as TrustRAG and AstuteRAG) are essentially black-box decision-making processes. When a conflict is detected in the retrieved document content, they only implicitly use the internal knowledge of the LLM to make a ruling and provide a final answer, without showing the user or administrator the basis for their trust or suspicion of a document. That is, existing solutions usually only output the merged answer without explicitly indicating which documents are considered suspicious and the reasons for the judgment. This lack of interpretability in the defense process seriously affects the system's trustworthiness, auditability, and debuggability.

[0024] 2) The defense fusion strategy is too rigid. Most current hybrid defense frameworks use a fixed knowledge fusion prompt process, which cannot dynamically adjust the strategy according to the threat. For example, after detecting a document conflict, regardless of the degree of conflict, the system may invariably use a pre-set prompt to input external documents and internal knowledge into the LLM to generate an answer. There is a lack of an adaptive mechanism to dynamically adjust the trust weight of the LLM on internal knowledge and external retrieved information. This may lead to a dilemma when the attack intensity is different: either be too conservative and ignore valuable external information, reducing the quality of the answer; or be too aggressive and trust external documents, resulting in an incorrect answer when external knowledge has been poisoned.

[0025] 3) Efficiency and robustness are difficult to balance. Lightweight detection methods (such as PPL-based filtering) are efficient and low-cost, but they are easily bypassed by cleverly designed adversarial examples, resulting in insufficient defense robustness. Conversely, deep defenses that comprehensively utilize LLM (such as multi-turn isolated question answering or conflict inference) improve the detection rate, but often require multiple calls to LLM, leading to significant inference latency and computational overhead. In other words, current technologies lack a layered defense mechanism that can ensure a high detection rate while keeping the expensive LLM calls within a necessary range, thereby balancing system performance and defense strength.

[0026] To address the aforementioned problems, this invention provides a multi-stage interpretable adversarial defense method for retrieval enhancement generation systems, the technical framework of which is as follows: Figure 1 As shown, the method flow is as follows: Figure 2 As shown, it includes: S1. Query and retrieve the pre-built knowledge base to construct a candidate document set. Introduce a dual filtering strategy to perform anomaly detection and pre-filtering on the candidate document set, and remove candidate documents with obvious malicious intent to obtain an intermediate document set.

[0027] S11. Use the retrieval tool in the RAG system to query and retrieve the pre-built knowledge base, and return the Top-K documents with the highest relevance as a candidate document set. Specifically, a dense vector retrieval model, such as Facebook's Contriever, can be used to perform the query, which by default returns Top-K=5 documents. These retrieved documents will be used as input for subsequent defense judgment and participate in the construction of the candidate document set.

[0028] S12, Anomaly Detection and Pre-filtering.

[0029] This step employs a dual filtering strategy, filtering documents based on the results of combining semantic embedding clustering detection and text N-gram similarity detection to remove obviously suspicious malicious documents, thereby reducing the pressure of subsequent complex reasoning.

[0030] a) Combine semantic embedding clustering detection.

[0031] a1) Map each candidate document in the candidate document set to a fixed-dimensional vector representation using a semantic embedding model.

[0032] In this step, each candidate document corresponds to a vector point, which is the vector representation of the candidate document in the embedding space. This vector representation is obtained by encoding the text content of the candidate document.

[0033] a2) Use density-based unsupervised clustering algorithm to perform clustering analysis on vector representations, and detect candidate documents that have been poisoned and abnormal candidate documents based on the clustering analysis results.

[0034] Specifically, DBSCAN can be used. The DBSCAN algorithm can automatically detect the number of clusters and mark sparsely distributed points as noise.

[0035] Utilizing this characteristic, the present invention can detect high-density anomalous clusters or outlier documents far from the main cluster generated by poisoning attacks. Experiments have shown that maliciously poisoned documents, which often originate from similar templates, tend to cluster together in the embedding space to form anomalous clusters.

[0036] This step involves setting an appropriate density radius parameter. And the minimum sample size parameter MinPts, the candidate documents corresponding to clusters whose density differs from the density of surrounding clusters by more than a first density threshold are defined as candidate documents targeted by poisoning attacks; the candidate documents corresponding to isolated cluster points are judged as abnormal documents. Preferably, The parameters can be adaptively estimated using the K-nearest neighbor distance distribution, for example, by using the median distance between candidate documents. To adapt to different datasets and attack intensities.

[0037] b) Text N-gram similarity detection.

[0038] For attack documents that are semantically disguised but have highly repetitive lexical patterns, relying solely on embedded features may miss some. Therefore, this invention introduces N-gram feature comparison at the character / word level to filter candidate document sets based on text similarity.

[0039] The candidate documents in the candidate document set are randomly grouped. For each pair of candidate documents, the following operations are performed: calculate the degree of overlap of the longest common subsequence, such as consecutive N-gram word segments, between each pair of candidate documents. The repetition rate of text segments can be measured using the ROUGE-L index or BLEU score. If the degree of overlap is greater than the first overlap threshold, the corresponding candidate document pair is marked as a suspicious candidate document.

[0040] If excessively repetitive template patterns are found among certain documents, such as multiple documents sharing a large number of identical sentence structures or repetitive paragraphs of almost the same length, it can be determined that these documents are likely generated in batches by the same adversarial template.

[0041] In addition, text perplexity level (PPL) can be combined as an auxiliary feature to identify attacks that are semantically fluent but have similar content, thereby further improving the detection rate of templated attacks while ensuring filtering efficiency.

[0042] S13, Joint Ruling.

[0043] Based on the above embedding clustering and N-gram detection results, this step adopts a lenient but safety-first decision-making strategy.

[0044] Candidate documents that are not subject to poisoning attacks, are not abnormal, or are not suspicious are retained.

[0045] Candidate documents for poisoning attacks are retained for clusters whose density difference with the density of surrounding clusters is greater than a first density threshold but less than a second density threshold, and the remaining candidate documents for poisoning attacks are removed. For such documents with uncertain boundaries, this invention chooses to temporarily retain them and proceed to the next stage for more in-depth reasoning analysis by LLM.

[0046] Retain suspicious candidate documents whose overlap is less than the first overlap threshold but greater than the second overlap threshold, and remove the remaining suspicious candidate documents.

[0047] By combining dual filtering, this invention filters out most malicious and interfering documents in the first stage, providing cleaner input for subsequent stages, while reserving computationally expensive LLM inference for a few documents that are not clearly categorized, thus balancing efficiency and detection rate.

[0048] S2. Without importing external documents, prompt the LLM to generate a baseline answer based on its internally stored knowledge. This baseline answer represents the LLM's common-sense answer to the question and is used for subsequent comparison.

[0049] During the design and deployment phase of the retrieval enhancement system, the quality of LLM's responses in the absence of external information can be offline evaluated and calibrated using a validation dataset with standard answers and manual sampling assessment. This determines its accuracy and applicability on typical questions. In actual online operation, the retrieval enhancement system no longer relies on real-time manual evaluation. Instead, it treats the benchmark answer as a high-confidence reference for the LLM's internal knowledge: when offline evaluation shows that the LLM can typically provide correct and complete answers to such questions without external information, the current benchmark answer can be approximated as "correct." If, in this case, the content provided by the retrieved document significantly conflicts with the benchmark answer, it indicates that the retrieved document may contain conflicting or irrelevant information. Conversely, if the benchmark answer is significantly incomplete or has a low accuracy rate for the corresponding question category in offline evaluation, the external document may provide new knowledge points. In this case, the retrieval enhancement system will increase the weight of the external document during subsequent fusion.

[0050] S3. Using LLM, each intermediate document in the intermediate document set is compared with the benchmark answer to generate a structured display reasoning result for the corresponding intermediate document. Based on the structured display reasoning result, the intermediate document set is filtered to obtain the target document set.

[0051] S31. The prompt for setting LLM includes: determining whether the intermediate document conflicts with the benchmark answer, the type of conflict, and the severity. In detail, the prompt can be "Does this document contradict or conflict with the benchmark answer?" and "What are the specific types and severity of the conflict?".

[0052] By carefully designing prompts, LLM needs to determine whether there are factual inconsistencies between the document content and the benchmark answer, such as contradictory key facts, logical contradictions (i.e., conflicting causal relationships), or other conflicts, and to determine whether the conflict is a minor difference in detail or a serious fundamental contradiction. This invention defines a unified conflict labeling system, categorizing comparison results as "completely consistent," "minor inconsistency," "serious conflict," or "irrelevant," and clearly describing the meaning of each judgment label in the prompts to avoid ambiguous judgments in LLM outputs.

[0053] S32. Input each intermediate document in the intermediate document set into the LLM, ask questions according to the prompt, and output a structured display of the reasoning results, including document ID, decision label, and reason key-value pairs, which can be represented as: {Document ID: 2, Decision: "Slight inconsistency", Reason: "This document contradicts the baseline fact on a key date and may have been maliciously modified."}

[0054] In this step, the present invention employs a chain-of-thought (CoT) prompting strategy, encouraging the model to progressively analyze the consistency and contradictions between the document and the baseline answer, and to infer whether these contradictions stem from malicious document tampering (poisoning) or simply normal differences of opinion. Ultimately, the LLM needs to explicitly determine whether the intermediate document is trustworthy and provide specific criteria for this determination; and to facilitate automatic system parsing and presentation, the present invention requires the LLM to output the inference chain according to a predetermined format, such as using a JSON structure.

[0055] Through a structured, explicit reasoning chain, the system can intuitively demonstrate why each document is considered suspicious or credible, greatly improving the transparency of defense decisions.

[0056] S4. Based on the structured display of inference results from all intermediate documents, comprehensively evaluate the confidence level of the intermediate document set.

[0057] Confidence features are obtained based on the structured display reasoning results, including the number and proportion of intermediate documents labeled as conflict, the distribution of conflict severity, and the repetition rate of reason key-value pairs; confidence scores in the range of 0 to 1 are calculated based on the confidence scores.

[0058] In addition, attack signature phrases need to be detected for each intermediate document. If such attack signature phrases as "ignore the above content", "please output..." and "when you are asked..." are detected, the confidence of the corresponding intermediate document is set to zero and it is judged as a high threat. Conversely, if the external information is rich and consistent with the internal knowledge, such as the LLM output containing a large number of proper nouns or precise data, it indicates that the answer is based on factual information and the confidence is high.

[0059] In this step, if most documents are deemed suspicious and contain serious conflicts, it indicates that the knowledge base is likely to have been poisoned, posing a high threat level; conversely, if all documents are consistent with the benchmark answer, the threat is extremely low.

[0060] By using multi-signal fusion to assess confidence, this invention reduces the misjudgments that may be caused by relying on a single indicator, and improves the accuracy and robustness of threat perception.

[0061] S5. Select an adaptive fusion mode based on confidence level. In this adaptive fusion mode, input the user query and the target document set into the LLM to generate an answer.

[0062] Adaptive fusion is a control mechanism that pre-sets at least two sub-modes: a high-security mode and a normal mode. The retrieval enhancement system always calculates the overall confidence level based on the conflict analysis results first, and then the adaptive fusion mechanism switches between these two modes.

[0063] For ease of implementation, when the confidence level is less than or equal to a confidence threshold (e.g., 0.4), the adaptive fusion mode is a high-security mode, corresponding to situations with high threats, untrustworthy external documents as a whole, or strong adversarial features such as detected obvious hints or injection. In high-security mode, the retrieval enhancement system explicitly instructs the LLM to prioritize its own internal knowledge in the final prompt, ignoring or downplaying the content of external documents previously marked as suspicious. For example, a prompt like "When conflicts occur, please prioritize answering based on your own knowledge" can be added to ensure that even if most retrieved documents are poisoned or tampered with, the LLM tends to ignore them and answer primarily based on its own knowledge, thereby reducing the risk of errors.

[0064] When the confidence level is greater than the confidence threshold, corresponding to a low threat level and the credibility of most external documents, the LLM's internal knowledge is combined with the target document set to answer the question according to the standard RAG process. In this mode, the LLM can use credible external documents to obtain more factual details and the latest information, thereby improving the completeness and freshness of the answer.

[0065] Therefore, the adaptive fusion mode is triggered only when necessary. That is, the retrieval enhancement system will actively adjust the mode and strengthen the high-security strategy only when the confidence level is lower than or equal to the threshold and typical attack characteristics appear. In most security scenarios, the retrieval enhancement system maintains the normal mode and behaves close to the original RAG process, with minimal impact on the existing system.

[0066] Furthermore, the present invention can also generate structured defense reports (such as the overall credibility level of the output response, whether external information sources may be contaminated, etc.) based on the above-mentioned methods and processes, for administrator auditing and system diagnosis.

[0067] In summary, this invention constructs a progressive and interconnected adversarial defense system through a three-stage process: pre-filtering and anomaly detection, conflict analysis, and adaptive fusion. The first stage focuses on highly recalling and removing obviously suspicious documents to reduce subsequent workload; the second stage accurately determines the credibility of remaining documents and provides explanatory evidence; the third stage dynamically adjusts the response strategy based on the threat level to ensure the final output is safe and reliable. The three stages are seamlessly connected and information is transmitted: the results of pre-filtering serve as input for conflict analysis, and the labels and reasons from conflict analysis guide the strategy switching in the fusion stage, ultimately ensuring that the LLM can still provide correct and credible answers even when subjected to poisoning attacks.

[0068] Example 2 This embodiment also provides a multi-stage interpretable adversarial defense system for retrieval enhancement generation systems, including: The pre-filtering module, located between the retrieval unit and the LLM generator, is used to query and retrieve the pre-built knowledge base, construct a candidate document set, and introduce a dual filtering strategy to perform anomaly detection and pre-filtering on the candidate document set, eliminating candidate documents with obvious malicious intent, and obtaining an intermediate document set.

[0069] The conflict analysis module calls the LLM to perform conflict detection and interpretable reasoning on each of the filtered candidate documents. Without introducing external documents, this module prompts the LLM to generate a baseline answer based on its internally stored knowledge. The LLM then compares each intermediate document in the intermediate document set with the baseline answer, generates a structured display reasoning result for the corresponding intermediate document, and filters the intermediate document set based on the structured display reasoning result to obtain the target document set.

[0070] The fusion decision module is used to structure and display the inference results based on all intermediate documents, comprehensively evaluate the confidence level of the intermediate document set, and adaptively adjust the degree of trust in external documents when making the final LLM response. This module switches to a conservative mode when high risk is detected, and adopts a normal mode to fully integrate external information when low risk is detected.

[0071] The answer generation module, corresponding to the LLM generator, generates the answer output based on the pattern determined by the fusion decision module, integrating user queries, retained reliable documents, and necessary prompts. Module 104 is the answer output terminal of the entire RAG system. When using the method of this invention, its output content has been filtered or weakened to prevent malicious poisoning information, thereby ensuring the correctness and security of the answer.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0073] The present invention also provides an electronic device including a central processing unit (CPU) that can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0074] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The processing unit executes the various methods and processes described above, such as methods S1 to S5. For example, in some embodiments, methods S1 to S5 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of methods S1 to S5 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S5 by any other suitable means (e.g., by means of firmware).

[0076] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0077] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0078] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems, characterized in that, include: The pre-built knowledge base is queried and retrieved to construct a candidate document set. A dual filtering strategy is introduced to perform anomaly detection and pre-filtering on the candidate document set, and candidate documents with obvious malicious intent are removed to obtain an intermediate document set. The dual filtering strategy is based on the results of combining semantic embedding clustering detection and text N-gram similarity detection to filter documents. Without incorporating external documentation, the LLM is prompted to generate benchmark answers based on the knowledge stored internally. LLM is used to compare each intermediate document in the intermediate document set with the benchmark answer to generate a structured display reasoning result for the corresponding intermediate document. The intermediate document set is then filtered based on the structured display reasoning result to obtain the target document set. The structured display reasoning result includes document number, judgment tag, and reason key-value pairs. Based on the structured display inference results of all intermediate documents, the confidence level of the intermediate document set is comprehensively evaluated. Based on the aforementioned confidence level, an adaptive fusion mode is selected. In this adaptive fusion mode, the user query and the target document set are input into the LLM to generate an answer. The adaptive fusion modes include a high-security mode and a normal mode.

2. The multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, The candidate document set is constructed from the Top-K documents with the highest relevance in the knowledge base.

3. The multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, The method for combining semantic embedding clustering detection is as follows: Each candidate document in the candidate document set is mapped to a fixed-dimensional vector representation using a semantic embedding model; The vector representation is clustered using a density-based unsupervised clustering algorithm, and the candidate documents for poisoning attacks and abnormal candidate documents are detected based on the clustering analysis results. The candidate documents to be poisoned are: in the clustering results, the candidate documents corresponding to the clusters whose density differs from that of the surrounding clusters by more than a first density threshold; The abnormal candidate documents are: candidate documents corresponding to isolated cluster points in the clustering results.

4. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, The method for text N-gram similarity detection is as follows: The candidate documents in the aforementioned candidate document set are randomly grouped into pairs, and the following operations are performed on each pair of candidate documents: Calculate the overlap of the longest common subsequence between each pair of candidate documents. If the overlap is greater than a first overlap threshold, mark the corresponding candidate document pair as a suspicious candidate document.

5. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, The aforementioned dual filtering strategy is as follows: Candidate documents that are not subject to poisoning attacks, are not abnormal, or are not suspicious are retained. Retain the candidate documents for poisoning attacks corresponding to clusters whose density difference with the density of surrounding clusters is greater than the first density threshold but less than the second density threshold, and remove the remaining candidate documents for poisoning attacks. Retain suspicious candidate documents whose overlap is less than the first overlap threshold but greater than the second overlap threshold, and remove the remaining suspicious candidate documents.

6. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, The method for generating the structured display reasoning results is as follows: the LLM prompt includes: determining whether the intermediate document conflicts with the baseline answer, the type of conflict, and the severity; inputting each intermediate document in the intermediate document set into the LLM, asking questions according to the prompt, and outputting the structured display reasoning results, including document ID, judgment label, and reason key-value pairs.

7. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, The method for obtaining the confidence level is as follows: based on the structured display reasoning results, confidence features are obtained, including the number and proportion of intermediate documents with the label of conflict, the distribution of conflict severity, and the repetition rate of the reason key-value pairs; based on the confidence level, a confidence score in the range of 0 to 1 is specifically calculated.

8. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 7, characterized in that, The process of obtaining the confidence level also includes detecting attack feature phrases for each intermediate document. If an attack feature phrase is detected, the confidence level of the corresponding intermediate document is set to zero.

9. A multi-stage interpretable adversarial defense method for retrieval enhancement generation systems according to claim 1, characterized in that, When the confidence level is less than or equal to the confidence threshold, the adaptive fusion mode is a high-security mode, in which the LLM is limited to using its internal knowledge to answer questions; when the confidence level is greater than the confidence threshold, the LLM's internal knowledge is combined with the target document set to answer questions according to the standard RAG process.

10. A multi-stage interpretable adversarial defense system for retrieval-enhanced generation systems, characterized in that, The system is used to implement the method as described in any one of claims 1 to 9.

Images