System member leakage risk assessment method based on retrieval enhancement of small amount of queries

Abstract

The application discloses a search enhancement generation system member leakage risk assessment method and system based on a small amount of queries. Under black box conditions, a target member document is used to generate a plurality of semantically equivalent and syntactically diverse queries by an auxiliary large language model, the queries are submitted to a target search enhancement generation system, and responses are collected; the responses are encoded into sentence vectors, the average pairwise similarity is calculated, and the penalty of the non-response response is identified, the final risk assessment score is synthesized, and compared with a threshold to evaluate the possibility of the target document as a knowledge base member. The application is implemented under complete black box conditions, does not depend on system internal architecture or parameter information, has low query overhead and is easy to calibrate, is suitable for compliance audit, red team evaluation and pre-online safety detection, so as to quantify the member privacy leakage risk and guide the protection strategy.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence system security and privacy protection, and relates to a method and system for assessing the risk of member leakage in a retrieval-enhanced generation system, specifically a method and system for assessing the risk of member leakage in a retrieval-enhanced generation system based on a small number of queries. Background Technology

[0002] Retrieval Enhancement Generation (RAG) systems improve generation quality by retrieving external knowledge bases and generating text responses based on the retrieval results, but also introduce the risk of privacy leakage from external knowledge bases. Existing research shows that by determining whether a specific target document D exists in the system's knowledge base C, document-level member reasoning detection can be achieved, which can be used to assess the member privacy leakage risk of retrieval enhancement generation systems. The core idea of ​​existing risk assessment detection methods is to utilize the behavioral differences exhibited by the model when it "knows" and "does not know" a document. These behavioral differences can be reflected in the generated text, the model's confidence, and even the response time. Such risk assessments are usually conducted under black-box conditions: the assessor can only submit queries to the RAG system's API and receive the generated text, but cannot access the model parameters, the retrieval implementation, or the knowledge base index. If the risk is proven to exist, it may lead to serious consequences. For example: (1) Leakage of trade secrets: determining whether a confidential technical specification is in the company's internal knowledge base. (2) Leakage of personal information: determining whether a patient's medical records are in a hospital's database. (3) Confirmation of model training data: confirming whether a copyrighted book has been used to build a knowledge base.

[0003] Existing methods for "risk assessment of member leakage in retrieval-enhanced generation systems" are mostly focused on academic research and have not yet formed a unified industrial assessment process. Common assessment paradigms include simulation testing based on benchmark datasets—systematically applying existing attack strategies in a controlled simulation environment and recording the judgment results to quantify the risk; and statistical probing methods based on black-box queries—these methods are closer to real-world operating scenarios, requiring no prior knowledge of the entire knowledge base, but rather observing the system response distribution through a large number of diverse queries to infer potential member leakage risks. Idealized white-box or gray-box assessments can provide more refined causal analysis and localization information during the research and development phase, but are usually difficult to widely adopt in production environments due to access and feasibility limitations. This invention aims to provide an assessment method with low query overhead and easy engineering integration under black-box conditions, for obtaining authorized security assessment and compliance detection results and guiding protection improvements.

[0004] While the methods described above provide us with evaluation ideas, they generally have serious flaws, resulting in high costs, low feasibility, or unreliable results in practical applications.

[0005] (1) The evaluation cost is extremely high and difficult to scale; both benchmarking and black-box analysis require tens of thousands of queries to commercial APIs, resulting in huge costs. A large number of serial queries are extremely time-consuming and cannot be used for fast and frequent risk assessment. Building a high-quality benchmark dataset (including member and non-member documents and corresponding queries) requires a lot of manual intervention or consumes LLM computing power, which is a cumbersome process.

[0006] (2) Poor generalization ability of the evaluation results; the results obtained on one RAG system cannot be directly transferred to another RAG system using a different retrieval tool, different embedding model, or different LLM. Any change to any component may invalidate the previous risk assessment. Evaluation based on a fixed benchmark may lead to "overfitting"—the system may only be able to defend against known benchmark attacks, but be completely defenseless against new attack methods that may emerge in the future.

[0007] (3) There is a lack of unified and standardized metrics; some use accuracy, some use AUC, and some use custom similarity thresholds. This makes it impossible to directly compare risk assessment results between different studies and different systems. What level of accuracy is considered "high risk"? There is currently no industry consensus. The assessment result is often a relative value, rather than a clear and actionable absolute risk score. Summary of the Invention

[0008] Existing detection or assessment methods often rely on a large number of queries, white-box information, or complex prompting strategies, resulting in high costs and impractical deployment. The purpose of this invention is to propose a general black-box detection method and system based on a small number of queries, aimed at assessing and quantifying the risk of member leakage. This method can effectively determine whether a target document may exist in the system knowledge base through only a small number of query operations, without relying on the system's internal architecture or parameter information, thereby helping developers, security teams, or compliance personnel discover potential privacy leakage risks.

[0009] The technical solution adopted by the method of the present invention is: a method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries, comprising the following steps: Step 1: Under completely black-box access conditions, generate several semantically consistent but syntactically different queries based on the target member document to be determined; Step 2: Submit the query to the target retrieval enhancement generation system and obtain the corresponding generated response; Step 3: Encode the generated response into a set of vectors using a pre-trained sentence embedding model; Step 4: Calculate the semantic similarity score based on the vector set; Step 5: Identify and detect avoidance or refusal responses, and include the detected avoidance or refusal responses in the refusal penalty; Step 6: Combine the semantic similarity score with the rejection penalty to form a final risk assessment score, and compare the final score with a preset threshold to determine the likelihood of the target member document being a member in the target retrieval enhancement generation system's knowledge base, thereby assessing the risk of the target retrieval enhancement generation system leaking member privacy.

[0010] As a preferred option, in step 1, the template of "example document fragment + several corresponding queries" is used to guide the generation of queries by the large language model. The generated queries must meet the following conditions: ensure exclusivity, answerability, semantic equivalence and syntactic diversity, and avoid using keywords or formats that may trigger the security strategy of the target retrieval enhancement generation system through automated verification and screening. The so-called exclusivity is to extract key elements from the target document fragment that can uniquely point to the document. First, named entity recognition is performed, then keywords with weights higher than a threshold are extracted, and finally the frequency of occurrence of keywords is calculated on a representative corpus, with the lowest frequency being taken as the key element. The aforementioned answerability, semantic equivalence, and syntactic diversity involve constructing multiple semantically equivalent but syntactically different queries for each query. First, several seed query templates are constructed based on extracted exclusive identifiers and the target sentence to ensure semantic clarity and targeted questions, rather than open-ended or ambiguous queries. Then, the auxiliary large model is explicitly required to "generate semantically equivalent but syntactically different" queries. The system first generates "queries" and provides examples to demonstrate syntactic diversity. Then, it uses post-processing methods to regularize syntactic transformations and replace synonyms to enhance diversity. Finally, it performs parallel candidate selection and deduplication on the generated queries to filter out the candidate set that meets the criteria. The automated verification and filtering process selects a number of queries that ultimately meet the criteria. First, a security and compliance filter is performed, matching and filtering the generated queries. If non-compliant content is found, the queries are discarded or replaced. Queries with lengths outside a preset range or containing special formats are also filtered. Then, the sentence vector similarity between the candidate queries and the original documents is calculated, and a parse tree is used to calculate the syntactic similarity. This ensures that the queries contain at least two exclusive identifiers, and that there are no ambiguous pronouns or words with ambiguity. The questions end with a clear question mark or a definite question intent, thus selecting the final queries.

[0011] Preferably, in step 3, the sentence embedding model includes an input embedding module, a lightweight self-attention encoding module, an attention gating module, a sentence vector pooling module, and a projection normalization module. The input embedding module is used to transform the input text sequence into a vector representation that can be processed by the model. This module includes a word segmentation unit, a word embedding layer, and a positional encoding layer. The word segmentation unit is used to divide the input text into several tokens. The word embedding layer is used to map each token into a low-dimensional dense vector. The positional encoding layer is used to add positional information to the low-dimensional dense vector to maintain the sequence order. The output of this module serves as the input to the subsequent lightweight self-attention encoding module. The lightweight self-attention encoding module consists of six sequentially stacked encoding layers, each employing a lightweight Transformer structure. Each layer includes a multi-head self-attention sub-layer, a feedforward network sub-layer, layer normalization, and residual connection structures. The multi-head self-attention sub-layer is used to capture the contextual dependencies between different tokens in a sentence. The feedforward network sub-layer consists of two linear transformation layers. Each sub-layer employs a Pre-LN layer normalization structure and retains residual connections to ensure training stability and representational capability. The attention gating module is embedded in the self-attention output of each coding layer and is used to dynamically adjust the importance of different attention channels. The module calculates a learnable gating coefficient for each attention head and multiplies the coefficient with the corresponding attention output to achieve channel-level feature selection. The sentence vector pooling module is used to convert the vector representations of all tokens into fixed-dimensional sentence-level embedding representations. This module adopts a weighted pooling mechanism based on attention weights, which determines the importance of each token in the semantics of the sentence by calculating the attention score of each token. Based on this, the token representations are weighted and averaged to obtain a semantically weighted global sentence vector. The projection normalization module includes a linear transformation layer and an L2 normalization layer; the linear transformation layer is used to map the sentence vector to the target embedding dimension space; the L2 normalization layer is used to normalize the sentence vector so that the subsequent similarity calculation scale is consistent; the vector output by this module is the final sentence embedding representation.

[0012] Preferably, in step 4, the semantic similarity score is the average pairwise similarity of the response vectors. in, For the first One response Embedded vector, Number of responses.

[0013] Preferably, in step 5, a set of rejection keywords is defined. and will refuse punishment The regulations are as follows:

[0014] in, The set is a preset fixed negative constant; By performing steps 1-2 on known non-member documents, we collect the avoidance or rejection responses of the target retrieval enhancement generation system, and extract common rejection statements and their equivalent expressions from them. Indicates any response Appearing in A keyword in [the text].

[0015] Preferably, in step 6, the final risk assessment score... ; Score semantic similarity. To avoid punishment; and before initiating a judgment against the target document, use a set of offline non-member documents and offline member documents against a preset fixed negative constant. With classification threshold Perform adjustment and calibration; when The target document is evaluated as member-oriented if it is, otherwise it is evaluated as non-member-oriented.

[0016] Preferably, before initiating the judgment on the target document, a set of offline member samples and offline non-member samples of sufficient size to represent the distribution of the target data are used to evaluate a preset fixed negative constant. With classification threshold To conduct a joint evaluation, firstly, representative member document sets and representative non-member document sets are randomly selected from the data distribution of the target retrieval enhancement generation system. The size of each set should be sufficient to cover the diversity of the target distribution. Then, for each sample document, query generation, response collection, vectorization, and similarity calculation are performed to obtain a semantic consistency score for each sample. And whether the set of keywords for refusing to answer is triggered The marker; then set the penalty value. A candidate search interval is defined, within which candidate values ​​are enumerated with a specific step size; a classification threshold is set. The candidate space is the final score of the sample. The effective interval is determined and traversed with an appropriate step size for refined selection; for each candidate... Calculate the corresponding samples The distribution is determined, and an ROC curve is plotted to calculate the AUC, which is used to compare different distributions. Overall distinguishing ability; simultaneous traversal The candidate values ​​are calculated. The TPR, FPR, Precision, and Recall metrics are used, and the threshold-related criteria are used to determine the... Optimal threshold Record each candidate Corresponding AUC, and other evaluation metrics; by comparing all candidates AUC, determine the optimal penalty and their corresponding thresholds To ensure the robustness of parameter selection, nested cross-validation was used to repeat the above steps, and the mean and confidence interval of AUC, TPR, and FPR at different folds were calculated to evaluate the stability and uncertainty of parameter selection.

[0017] Preferably, the target member documents must first meet the conditions of length and topic concentration. The length should be within a preset range and the document content should revolve around one or a few topics rather than fragmented multiple topics. The topic concentration is evaluated using a clustering method based on sentence or paragraph embedding. Secondly, the documents should contain relatively unique facts, terms or expressions to improve the ability of the generated query to detect document features. The relatively unique facts and terms are weighted and synthesized using three metrics: term frequency-inverse document frequency (IDF), named entity uniqueness, and low-frequency n-gram ratio. The average IDF is calculated from the document frequency of the representative corpus. Entity uniqueness is measured by extracting document entities and counting their frequency of occurrence in the representative corpus to determine the proportion of low-frequency entities. The low-frequency n-gram ratio is obtained by counting the number of occurrences of all three-grams in the document in the representative corpus and comparing them with a preset frequency threshold. The above three criteria are combined into a comprehensive uniqueness score according to a predetermined weight and compared with a preset threshold. When the comprehensive uniqueness score reaches or exceeds the threshold, it is considered to "contain relatively unique facts or terms".

[0018] This invention also provides a retrieval enhancement generation system based on a small number of queries for member leakage risk assessment, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries.

[0019] The present invention also provides a retrieval enhancement generation system member leakage risk assessment product based on a small number of queries, including computer program instructions, which, when run on a computer, cause the computer to execute the retrieval enhancement generation system member leakage risk assessment method based on a small number of queries.

[0020] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows: (1) This method requires only a small number of queries to complete the risk assessment, which significantly reduces the time and computing resource overhead in the assessment process; compared with traditional methods that rely on hundreds or even thousands of queries, the efficiency is improved by several times or even tens of times, and it is more suitable for routine testing and pre-launch auditing of large-scale systems; (2) This method can be implemented under completely black-box conditions, without relying on the system's internal model architecture, parameter weights, or index data access permissions, and therefore can be widely applied to various real-world deployed retrieval enhancement generation systems. This makes the evaluation process closer to real attack scenarios, thereby improving the reference value of the evaluation results; (3) By combining semantic consistency analysis and rejection penalty mechanism, this method can effectively distinguish between member documents and non-member documents. Experiments show that this method achieves good results in various combinations of retrieval and generator methods, significantly outperforming the comparison methods; (4) The probe queries generated by this method have syntactic diversity under the premise of semantic equivalence, avoiding suspicious inputs with repetitive templates, thus having good concealment; at the same time, it still maintains strong robustness and avoidance ability when facing defense mechanisms such as semantic awareness detection and query rewriting. (5) This method provides interpretable parameters and can be flexibly calibrated using offline samples, making it easy to integrate into existing security assessment and red team processes. Its simple and clear implementation also facilitates rapid deployment by developers in practice. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram illustrating the method principle of an embodiment of the present invention; Figure 2 This is a network structure diagram of the sentence embedding model according to an embodiment of the present invention. Detailed Implementation

[0023] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] This invention aims to provide developers of artificial intelligence systems, privacy compliance departments, and security researchers with a tool to detect and quantify the risk of member privacy breaches. It can reveal potential vulnerabilities in data security and user privacy of retrieval augmentation systems and provide a reference for the design of subsequent defense measures. In this way, while promoting the application of large language models and retrieval augmentation technologies, it can enhance user privacy protection and system credibility.

[0025] Please see Figure 1 This embodiment provides a method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries, characterized by the following steps: Step 1: Under the condition of complete black-box access, use the auxiliary large language model to generate several semantically consistent and syntactically different queries based on the target member document to be determined; In one implementation, a template of "example document fragment + several corresponding queries" is used to instruct the auxiliary large language model to generate queries. The generated queries must meet the following conditions: ensuring exclusivity, answerability, semantic equivalence, and syntactic diversity, and avoiding the use of keywords or formats that might trigger the security strategy of the target retrieval enhancement generation system. The implementation of these conditions consists of three main modules. The first module is document feature extraction, which aims to extract key elements from the target document fragment that uniquely point to the document, such as entities, proper nouns, special phrases, and numbers. This involves first performing named entity recognition, then extracting keywords with high weights, and finally calculating the frequency of keyword occurrence on a representative corpus; the lower the frequency, the more unique the keyword. This module ensures exclusivity. The second module is query generation, which aims to construct multiple semantically equivalent but syntactically different queries for each query. First, several seed query templates are constructed based on the extracted exclusivity identifier and the target sentence to ensure that the semantics are clear and the question is targeted, rather than an open-ended or ambiguous question. Then, the auxiliary large model is explicitly required to "generate semantically equivalent but syntactically different queries." The first module generates a set of queries and provides examples to demonstrate syntactic diversity. Secondly, post-processing methods, such as back-translation, regularized syntactic transformation, and synonym replacement, are used to enhance diversity. Finally, parallel candidate generation and deduplication are performed to filter out the candidate set that meets the criteria. This module ensures answerability, semantic equivalence, and syntactic diversity. The third module is automated validation and filtering, aiming to select a final set of queries that meet the criteria. First, security compliance filtering is performed, matching and filtering the generated queries; queries containing sensitive words, URLs, or code snippets are discarded or replaced. Queries that are too short, too long, or contain special formats are also filtered. Then, the sentence vector similarity between the candidate queries and the original documents is calculated, and parse trees are used to calculate syntactic similarity. This ensures that the query contains at least two exclusive identifiers, has no ambiguous pronouns or ambiguities, and ends with a clear question mark or a definite question intent, thus filtering out the final queries. This module ensures exclusivity, answerability, semantic equivalence, and syntactic diversity, and guarantees that the target retrieval enhancement generation system's security policy detection is passed.

[0026] In one embodiment, the auxiliary large language model may be o4-mini; the listed model names and combinations are for illustrative purposes only and are not intended to limit the scope of the claims.

[0027] Step 2: Submit the query to the target retrieval enhancement generation system and obtain the corresponding generated response; Step 3: Encode the generated response into a set of vectors using a pre-trained sentence embedding model; In one implementation, the sentence embedding model includes an input embedding module, a lightweight self-attention encoding module, an attention gating module, a sentence vector pooling module, and a projection normalization module.

[0028] The input embedding module is used to transform the input text sequence into a vector representation that can be processed by the model. This module includes a word segmentation unit, a word embedding layer, and a positional encoding layer. The word segmentation unit is used to divide the input text into several tokens. The word embedding layer is used to map each token into a low-dimensional dense vector. The positional encoding layer is used to add positional information to the embedding vector to maintain the sequence order. The output of this module serves as the input to the subsequent lightweight self-attention encoding module. The lightweight self-attention encoding module consists of six sequentially stacked encoding layers, each employing a lightweight Transformer structure. Each layer includes a multi-head self-attention sub-layer, a feedforward network sub-layer, layer normalization, and residual connection structures. Specifically: the multi-head self-attention sub-layer captures the contextual dependencies between different tokens in a sentence; compared to the traditional Transformer model, the number of attention heads is reduced from 12 to 8, thus reducing computational complexity; the feedforward network sub-layer consists of two linear transformation layers, reducing the hidden dimension expansion ratio from 4 times to 2 times, achieving model lightweighting; each sub-layer employs a Pre-LN layer normalization structure and retains residual connections to ensure training stability and representational capability. The attention gating module is embedded in the self-attention output of each encoding layer to dynamically adjust the importance of different attention channels; this module calculates a learnable gating coefficient for each attention head. ,in This represents a mapping function used to map the output of a linear transformation to a suitable numerical range. It is the first The linear weights corresponding to each attention head; The input feature vector represents the gate and the basis for determining the gate. The bias term is used to shift the result of the linear mapping before activation, and the coefficient is multiplied by the corresponding attention output to achieve channel-level feature selection. In this way, the model can automatically activate or suppress different attention channels according to the input features, thereby improving information utilization efficiency and reducing invalid computation. The sentence vector pooling module is used to convert the vector representations of all tokens into fixed-dimensional sentence-level embedding representations. This module adopts a weighted pooling mechanism based on attention weights: by calculating the attention score of each token, its importance in the semantics of the sentence is determined; and the token representations are weighted and averaged accordingly to obtain a semantically weighted global sentence vector. Compared with traditional average pooling or [CLS] vector methods, this module can better highlight key information in the sentence. The projection normalization module includes a linear transformation layer and an L2 normalization layer; the linear transformation layer is used to map the sentence vector to the target embedding dimension space; the L2 normalization layer is used to normalize the sentence vector so that the subsequent similarity calculation scale is consistent; the vector output by this module is the final sentence embedding representation, which can be used for text retrieval, semantic matching or generation tasks. The sentence embedding model employs a lightweight distillation training strategy: during the training phase, knowledge transfer is achieved by aligning the last two layers of attention distribution of the teacher model with the sentence vector output representation; at the same time, a contrastive learning loss function is used to make the distance between synonymous sentence vectors closer and the distance between heteronymous sentences farther; further combined with quantized perceptual training, the deployment performance of the model on low-bit hardware platforms is improved.

[0029] In one implementation, the sentence embedding model may also be an existing all-MiniLM-L6-v2, or a functionally equivalent sentence embedding model (e.g., a model based on the Sentence-BERT or Sentence-Transformers architecture); the model names are listed only as implementation examples and are not intended to limit the scope of the claims.

[0030] Step 4: Calculate the semantic similarity score based on the vector set; In one implementation, the semantic similarity score is the average pairwise similarity of the response vectors. in, For the first One response Embedded vector, Number of responses.

[0031] Step 5: Identify and detect avoidance or refusal responses, and include the detected avoidance or refusal responses in the refusal penalty; In one implementation, a set of rejection keywords is defined. and will refuse punishment The regulations are as follows:

[0032] in, The set is a preset fixed negative constant; By performing steps 1-2 on known non-member documents, we collect the avoidance or rejection responses of the target retrieval enhancement generation system, and extract common rejection statements and their equivalent expressions from them. Indicates any response Appearing in A keyword in [the text].

[0033] Step 6: Combine the semantic similarity score with the rejection penalty to form a final risk assessment score, and compare the final score with a preset threshold to determine the likelihood of the target member document being a member in the target retrieval enhancement generation system's knowledge base, thereby assessing the risk of the target retrieval enhancement generation system leaking member privacy.

[0034] In one implementation, the final risk assessment score Before initiating a judgment against the target document, a set of offline non-member documents and offline member documents are used to set a preset fixed negative constant. With classification threshold Perform adjustment and calibration; when If the condition is met, the individual is considered a member; otherwise, they are considered a non-member.

[0035] In one implementation, in order to calibrate the fixed negative constant (Penalty value) and classification threshold To verify the performance of this method, a set of offline member samples and offline non-member samples of sufficient size to represent the target data distribution can be used for parameter tuning and evaluation. For example, 500 documents existing in the target knowledge base can be randomly selected as member samples, and another 500 documents from the same data distribution but not in the target knowledge base can be selected as non-member samples for offline calibration and performance evaluation. The 500 / 500 ratio is an exemplary value and is not intended to limit the scope of this claim. First, 500 representative member documents and 500 representative non-member documents are randomly selected from the data distribution of the target system. The size of each set should be sufficient to cover the diversity of the target distribution. The query generation—response acquisition—vectorization—similarity calculation process of this invention is executed on each sample document to obtain the semantic consistency score of each sample. And whether the set of keywords for refusing to answer is triggered The marker. Then set the penalty value. A candidate search interval is defined, within which candidate values ​​are enumerated with a specific step size. A classification threshold is set. The candidate space is the final score of the sample. The effective interval is determined and traversed with an appropriate step size for refined selection. For each candidate... Calculate the corresponding samples The distribution is determined, and an ROC curve is plotted to calculate the AUC, which is used to compare different distributions. Overall distinguishing ability; simultaneous traversal The candidate values ​​are calculated. The metrics include TPR, FPR, Precision, and Recall, and the determination of these metrics is based on threshold-related criteria. Optimal threshold Record each candidate Corresponding AUC, And other evaluation metrics. By comparing all candidates AUC, determine the optimal penalty and their corresponding thresholds To ensure the robustness of parameter selection, nested cross-validation was used to repeat the above steps, and the means and confidence intervals of AUC, TPR, and FPR at different folds were calculated to assess the stability and uncertainty of parameter selection. If necessary, a stable cross-validation approach can be selected. Take a more conservative value within the range, or use a soft penalty or weighted penalty strategy to improve robustness.

[0036] In one implementation, the target member document must first meet the conditions of length and topic concentration. The length should be within a preset range, and the document content should revolve around one or a few topics rather than fragmented multi-topic content. The topic concentration can be evaluated using a clustering method based on sentence or paragraph embedding. Secondly, the document should contain relatively unique facts, terms, or expressions (rather than common sense or highly generalized expressions) to improve the ability of the generated query to detect document features. To quantify this "relative uniqueness," this invention uses three metrics—term frequency-inverse document frequency (IDF), named entity uniqueness, and low-frequency n-gram ratio—and performs a weighted synthesis: the average IDF is calculated from the document frequency of the representative corpus; entity uniqueness is measured by extracting document entities and counting their frequency of occurrence in the representative corpus to measure the proportion of low-frequency entities; the low-frequency n-gram ratio is obtained by counting the number of occurrences of all three-grams in the document in the representative corpus and comparing them with a preset frequency threshold. The above three criteria are combined into a comprehensive uniqueness score according to a predetermined weight, and compared with a preset threshold. When the comprehensive uniqueness score reaches or exceeds the threshold, it is considered to "contain relatively unique facts or terms". Based on the above requirements, the generated query can more easily detect document features.

[0037] The invention will be further illustrated below through specific experiments.

[0038] The experiment first constructs a test environment, namely, building a RAG black-box test instance for each dataset, treating selected documents as either members of the knowledge base or excluding them as non-members, and testing the retrieval and generator combinations one by one. First, for each target document... Several short fragments are extracted as input, and the auxiliary large model o4-mini is used to generate four semantically consistent but syntactically diverse probe queries according to the template of "example document fragment + 4 corresponding queries". After generation, offline non-member samples are used for preliminary filtering / calibration; then, the above queries are submitted to the target RAG system one by one to collect the corresponding generated answers. The RAG retrieval process uses a pre-trained encoder to map documents to a vector library and retrieves the top-5. Finally, the responses are processed using the SentenceTransformer sentence vector model to calculate the average pairwise cosine similarity (Avg_sim) after removing the diagonal, and to detect whether the responses contain the set of rejected keywords. (e.g., "don't know", "not found"), if these occur, a fixed negative penalty will be imposed. The final score was During the offline phase, parameter tuning is performed using known non-member samples; comparisons are then made during runtime. Use thresholds to assess the risk of member leakage.

[0039] This experiment is divided into three stages: In the first stage, namely the query generation stage, we first start from the target document. Key fragments are extracted from the document to fully express its core content. Based on these fragments, a series of semantically consistent but syntactically diverse queries are generated. Specifically, a large auxiliary language model is used to generate four distinct queries. Each query is carefully crafted to be highly specific to the target document and can only be answered accurately if the document actually exists in the knowledge base. While these queries cover the same topic, they differ syntactically, which helps avoid detection based on repetitive defense mechanisms and ensures that the queries appear natural and diverse. This structural diversity improves the robustness of risk assessment when the target document exists in the knowledge base, preventing the model from relying on a single query format.

[0040] The second stage is response retrieval. In this stage, the generated query is submitted to the target retrieval enhancement generation system. The retrieval enhancement generation system processes each query, generating an answer by retrieving relevant documents from the knowledge base. The system ranks the retrieved documents based on the semantic similarity between the query and the documents, and combines these documents with the original query to generate a response according to a predetermined format. By analyzing the response from the retrieval enhancement generation system, the target document can be identified. Is it in the knowledge base? If the target document exists, the system's responses to all four queries should be highly consistent semantically; if the document does not exist, the system's responses will differ significantly, indicating that the document is not a member.

[0041] In the third stage, the membership inference stage, semantic similarity between the responses of all queries is evaluated using advanced sentence embedding techniques. This technique measures the degree of match between the system response and the expected output of the target document content. To improve the accuracy of risk assessment, a rejection penalty mechanism is introduced to penalize responses containing ambiguous or rejection terms to prevent erroneous negative sample judgments. Finally, the calculated similarity score is compared with a preset threshold. If the score exceeds the threshold, the target document is determined to be membership-biased, meaning the document actually exists in the knowledge base of the retrieval enhancement generation system; otherwise, it is determined to be non-membership-biased. Furthermore, known non-member documents can be used to adjust the threshold and rejection penalty factor offline to optimize the accuracy of risk assessment.

[0042] Experimental results show that in control experiments using three real-world corpora (NFCorpus, TREC-COVID, SCIDOCS) and two retrieval tools (Contriever, Contriever-MSMARCO) and four generators (Qwen2.5-7B-Instruct, Gemini-2.0-Flash, GPT-4o-mini, and Deepseek-v3), this method demonstrates stable and excellent performance in an offline-calibrated 500 / 500 member / non-member setting, with an overall AUC typically ≥ 0.98 and an overall accuracy typically ≥ 0.94. Taking NFCorpus + Contriever + Qwen2.5-7B as an example, the AUC is 0.998, accuracy ≈ 0.975, and TPR ≈ 0.988 at FPR = 0.01. Compared to methods requiring a large number of queries, this method achieves or surpasses the performance of multi-round methods with only 4 probe queries, resulting in a query overhead reduction of approximately 7.5×. Furthermore, ablation experiments show that performance significantly improves (ROC increases from ≈0.92 to ≈0.97) when the number of queries n increases from 2 to 4, while exhibiting low sensitivity to the top-k retrievals (even at k=20, AUC remains ≈0.98). In adversarial protection evaluations, this method demonstrates good robustness and high stealth under tests with window-based semantic-aware rate limiting, common query rewriting strategies, and various context / leakage detectors: the average AUC against multiple automatic rewriting strategies remains low. 0.98, Accuracy The detection rate was 0.94; under the five types of context detectors used in the paper and the default threshold, the detection rate was close to 0%, while the control method with high query volume was significantly detected. These results demonstrate that the present invention can effectively and covertly assess the member disclosure risk of RAG systems in both low query budget and adversarial protection environments.

[0043] This invention's method, through a small number of carefully designed queries, can significantly improve the robustness and resistance to interference of large language models when facing member disclosure attacks, while ensuring high efficiency and avoiding triggering traditional defense mechanisms. This method does not rely on retraining the target large language model; instead, it achieves member inference of target documents through flexible query generation and feature analysis, reducing the complexity and cost of technical implementation. As industry knowledge changes and updates, the local knowledge base can be dynamically updated as needed, ensuring the timeliness and accuracy of the retrieval enhancement generation system when handling professional issues in different fields. By employing a small number of efficient queries, the risk of detection defense mechanisms is reduced, the success rate of member inference risk assessment is improved, and the defense risks caused by excessive adversarial queries are avoided.

[0044] It should be understood that the embodiments described above are only some, not all, of the embodiments of the present invention. Furthermore, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0045] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries, characterized in that, Includes the following steps: Step 1: Under completely black-box access conditions, generate several semantically consistent but syntactically different queries based on the target member document to be determined; The template of "example document fragment + several corresponding queries" is used to assist the large language model in generating queries. The generated queries must meet the following conditions: ensure exclusivity, answerability, semantic equivalence and syntactic diversity, and avoid using keywords or formats that may trigger the security strategy of the target retrieval enhancement generation system through automated verification and screening. The so-called exclusivity is to extract key elements from the target document fragment that can uniquely point to the document. First, named entity recognition is performed, then keywords with weights higher than a threshold are extracted, and finally the frequency of occurrence of keywords is calculated on a representative corpus, with the lowest frequency being taken as the key element. The aforementioned answerability, semantic equivalence, and syntactic diversity involve constructing multiple semantically equivalent but syntactically different queries for each query. First, based on extracted exclusive identifiers and the target sentence, several seed query templates are constructed to ensure semantic clarity and targeted questions, rather than open-ended or ambiguous queries. Then, the auxiliary large model is explicitly required to "generate semantically equivalent but syntactically different" queries. The system generates a query and provides examples to demonstrate syntactic diversity. It then uses back-translation, regularized syntactic transformations, and synonym replacement to enhance diversity. Finally, it performs parallel candidate selection and deduplication on the generated queries to filter out a candidate set that meets the criteria. The automated verification and filtering process selects a number of queries that ultimately meet the criteria. First, a security and compliance filter is performed, matching and filtering the generated queries. If non-compliant content is found, the queries are discarded or replaced. Queries with lengths outside the preset range or containing special formats are also filtered. Then, the sentence vector similarity between the candidate queries and the original documents is calculated, and a parse tree is used to calculate the syntactic similarity. This ensures that the queries contain at least two exclusive identifiers, and that there are no ambiguous pronouns or words with ambiguity. The questions end with a clear question mark or a definite question intent, thus filtering out the final queries. Step 2: Submit the query to the target retrieval enhancement generation system and obtain the corresponding generated response; Step 3: Encode the generated response into a set of vectors using a pre-trained sentence embedding model; Step 4: Calculate the semantic similarity score based on the vector set; The semantic similarity score is the average pairwise similarity of the response vectors. in, For the first One response Embedded vector, For the number of responses; Step 5: Identify and detect avoidance or refusal responses, and include the detected avoidance or refusal responses in the refusal penalty; Among them, the set of rejection keywords is defined. and will refuse punishment The regulations are as follows: in, A preset fixed negative constant; set By performing steps 1-2 on known non-member documents, the avoidance or rejection responses of the target retrieval enhancement generation system are collected, and common rejection statements and their equivalent expressions are extracted from them. Indicates any response Appearing in A keyword in; Step 6: Combine the semantic similarity score with the rejection penalty to form a final risk assessment score, and compare the final risk assessment score with a preset threshold to determine the likelihood of the target member document being a member in the target retrieval enhancement generation system's knowledge base, thereby assessing the risk of the target retrieval enhancement generation system leaking member privacy. Among them, the final risk assessment score ; Score semantic similarity. To avoid punishment; and before initiating a judgment against the target document, use a set of offline non-member documents and offline member documents against a preset fixed negative constant. With classification threshold Perform adjustment and calibration; when The target document is evaluated as member-oriented if it is, otherwise it is evaluated as non-member-oriented.

2. The method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries as described in claim 1, characterized in that: In step 3, the sentence embedding model includes an input embedding module, a lightweight self-attention encoding module, an attention gating module, a sentence vector pooling module, and a projection normalization module. The input embedding module is used to transform the input text sequence into a vector representation that can be processed by the model. This module includes a word segmentation unit, a word embedding layer, and a positional encoding layer. The word segmentation unit is used to divide the input text into several tokens. The word embedding layer is used to map each token into a low-dimensional dense vector. The positional encoding layer is used to add positional information to the low-dimensional dense vector to maintain the sequence order. The output of this module serves as the input to the subsequent lightweight self-attention encoding module. The lightweight self-attention encoding module consists of six sequentially stacked encoding layers, each employing a lightweight Transformer structure. Each layer includes a multi-head self-attention sub-layer, a feedforward network sub-layer, layer normalization, and residual connection structures. The multi-head self-attention sub-layer is used to capture the contextual dependencies between different tokens in a sentence. The feedforward network sub-layer consists of two linear transformation layers. Each sub-layer employs a Pre-LN layer normalization structure and retains residual connections to ensure training stability and representational capability. The attention gating module is embedded in the self-attention output of each coding layer and is used to dynamically adjust the importance of different attention channels. The module calculates a learnable gating coefficient for each attention head and multiplies the coefficient with the corresponding attention output to achieve channel-level feature selection. The sentence vector pooling module is used to convert the vector representations of all tokens into fixed-dimensional sentence-level embedding representations. This module adopts a weighted pooling mechanism based on attention weights, which determines the importance of each token in the semantics of the sentence by calculating the attention score of each token. Based on this, the token representations are weighted and averaged to obtain a semantically weighted global sentence vector. The projection normalization module includes a linear transformation layer and an L2 normalization layer; the linear transformation layer is used to map the sentence vector to the target embedding dimension space; the L2 normalization layer is used to normalize the sentence vector so that the subsequent similarity calculation scale is consistent; the vector output by this module is the final sentence embedding representation.

3. The method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries as described in claim 1, characterized in that: Before initiating a judgment on the target document, a set of offline member samples and offline non-member samples of sufficient size to represent the target data distribution are used to evaluate a preset fixed negative constant. With classification threshold Conduct a survey and evaluation; First, representative member document sets and representative non-member document sets are randomly selected from the data distribution of the target retrieval enhancement generation system. The size of each set should be sufficient to cover the diversity of the target distribution. For each sample document, query generation, response collection, vectorization, and similarity calculation are performed to obtain the semantic consistency score of each sample. And whether the set of keywords for refusing to answer is triggered The marker; then set the penalty value. A candidate search interval is defined, within which candidate values ​​are enumerated with a specific step size; a classification threshold is set. The candidate space is the final score of the sample. The effective interval is determined and traversed with an appropriate step size for refined selection; for each candidate... Calculate the corresponding samples The distribution is determined, and an ROC curve is plotted to calculate the AUC, which is used to compare different distributions. Overall distinguishing ability; simultaneous traversal The candidate values ​​are calculated. The TPR, FPR, Precision, and Recall metrics are used, and the threshold-related criteria are used to determine the... Optimal threshold Record each candidate Corresponding AUC, and other evaluation metrics; by comparing all candidates AUC, determine the optimal penalty and their corresponding thresholds To ensure the robustness of parameter selection, nested cross-validation was used to repeat the above steps, and the mean and confidence interval of AUC, TPR, and FPR at different folds were calculated to evaluate the stability and uncertainty of parameter selection.

4. The method for assessing the risk of member leakage in a retrieval enhancement generation system based on a small number of queries according to any one of claims 1-3, characterized in that: The target member documents must first meet the conditions of length and topic concentration. The length is within the preset range and the document content should revolve around one or a few topics rather than fragmented multiple topics. The topic concentration is evaluated by a clustering method based on sentence or paragraph embedding. Secondly, the documents should contain relatively unique facts, terms, or expressions to improve the ability of the generated queries to detect document features; The relatively unique facts and terms are weighted and synthesized using three metrics: term frequency-inverse document frequency, named entity uniqueness, and low-frequency n-gram ratio. The average term frequency-inverse document frequency is calculated from the document frequency of the representative corpus. Entity uniqueness is measured by extracting document entities and counting their frequency of occurrence in the representative corpus to measure the proportion of low-frequency entities. The low-frequency n-gram ratio is obtained by counting the number of occurrences of all three-grams in the document in the representative corpus and comparing them with a preset frequency threshold. The above three metrics are combined into a comprehensive uniqueness score according to a predetermined weight and compared with a preset threshold. When the comprehensive uniqueness score reaches or exceeds the threshold, it is considered to "contain relatively unique facts or terms".

5. A risk assessment system for member leakage in a retrieval enhancement generation system based on a small number of queries, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method for assessing the risk of member leakage of a retrieval enhancement generation system based on a small number of queries as described in any one of claims 1 to 4.

6. A member leakage risk assessment product for a retrieval enhancement generation system based on a small number of queries, comprising computer program instructions, characterized in that: When the computer program instructions are executed on a computer, the computer performs the method for assessing the risk of member leakage of a retrieval enhancement generation system based on a small number of queries as described in any one of claims 1 to 4.

Images