A privacy risk auditing and control method and system for large language models

Abstract

The application discloses a privacy risk auditing and control method and system for large language models, and solves the technical problem that the existing privacy risk auditing and control method for large language models is difficult to adapt to large-scale input and actual deployment scenarios. The method comprises the following steps: obtaining a to-be-audited text data set, processing and generating an initial query through a memory induction model, inputting the target large language model to obtain an initial candidate output; calculating an initial privacy risk score, screening the target score, and iteratively fine-tuning the memory induction model to obtain an optimized version. Then, based on the optimized model, a final query is generated, the target large language model is inputted to obtain a final candidate output, and a privacy risk comparison reference set is constructed. When receiving user input text, privacy compliance candidate screening, privacy-utility dual compliance candidate subset construction are sequentially completed, and finally, candidate searching is performed in combination with the subset and the user input text, and a final reply for the user is outputted.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for privacy risk auditing and control of large language models. Background Technology

[0002] Large Language Models (LLMs), trained on massive amounts of text corpora, possess powerful text generation and logical reasoning capabilities. However, during training, they develop inherent model memory characteristics: triggered by specific input prompts, the model can generate text fragments that are completely identical or highly similar to the original training data, directly leading to core risks such as data privacy leaks and loss of content compliance control. Existing research has systematically confirmed the typical phenomenon of large language models "outputting training memory content under adapted prompts," and clarified the correlation between the degree of model memory leakage and core parameters such as model size and context window length, laying the foundation for basic research understanding of the privacy risks of large language models.

[0003] With technological advancements, large language models with enhanced reasoning capabilities for complex tasks are gradually being implemented. Explicit reasoning mechanisms based on chained thinking and reasoning trajectories have become a core design element of mainstream systems. Recent cutting-edge research further indicates that privacy leaks in large language models are no longer limited to the final generated results; explicit reasoning processes such as reasoning trajectories and intermediate thought chains also pose a high probability of privacy leaks. Moreover, as the reasoning budget increases (increased reasoning steps and expanded candidate generation paths), the scope and probability of model memory exposure are further amplified.

[0004] Existing methods for privacy risk auditing and control for large language models mostly rely on constructing prompts for each individual sample or conducting multiple rounds of search / optimization. The detection cost increases linearly with the number of samples. This not only makes it difficult to achieve a unified and repeatable privacy risk assessment in scenarios with large amounts of input or complex applications, but also makes it impossible to uniformly quantify and assess the reproducible and reconstructed content in the model output caused by model memory. As a result, it is difficult to adapt to large-scale input and actual deployment scenarios. Summary of the Invention

[0005] This invention provides a method and system for privacy risk auditing and control for large language models, which solves the technical problem that existing privacy risk auditing and control methods for large language models are difficult to adapt to large-scale input and actual deployment scenarios.

[0006] The first aspect of this invention provides a privacy risk auditing and control method for large language models, comprising:

[0007] Obtain the text dataset to be audited, and use the memory-induced model to process the text dataset to generate multiple initial queries. Use each initial query as the input of the target large language model to obtain multiple initial candidate outputs.

[0008] Calculate the initial privacy risk score corresponding to each initial candidate output, select multiple target privacy risk scores from each initial privacy risk score, and perform multiple rounds of iterative fine-tuning on the memory-inducing model to obtain the optimized memory-inducing model;

[0009] Multiple final queries are generated based on the optimized memory-inducing model and input into the target large language model to obtain multiple final candidate outputs. A privacy risk comparison reference set is constructed based on the multiple final candidate outputs.

[0010] When user input text is received, the target large language model is used to perform privacy compliance candidate filtering based on the user input text and the privacy risk comparison reference set, and output multiple privacy compliance candidates.

[0011] A privacy-utility dual compliance candidate subset is constructed based on multiple privacy compliance candidates using a preset utility scoring rule, thereby generating a privacy-utility dual compliance candidate subset;

[0012] Based on the privacy-utility dual compliance candidate subset and the user input text, a candidate search is performed, and the final response is output to the user.

[0013] Optionally, the text processing of the text dataset to be audited using a memory-induced model generates multiple initial queries. Each initial query is then used as input to the target large language model to obtain multiple initial candidate outputs, including:

[0014] Each text containing potential privacy risks in the text dataset to be audited is segmented to obtain multiple prefix and suffix sample pairs;

[0015] The prefix in each of the prefix and suffix sample pairs is used as the input of the memory induction model, and the initial prompt word corresponding to each prefix is ​​output.

[0016] By concatenating each of the aforementioned prefixes with the corresponding initial prompt words, multiple initial queries can be obtained;

[0017] Each of the initial queries is used as input to the target large language model to obtain multiple initial candidate outputs.

[0018] Optionally, the step of calculating an initial privacy risk score corresponding to each initial candidate output, filtering multiple target privacy risk scores from each initial privacy risk score, and performing multiple rounds of iterative fine-tuning on the memory-inducing model to obtain an optimized memory-inducing model includes:

[0019] A similarity calculation is performed on each initial candidate output and its corresponding suffix to obtain multiple initial privacy risk scores;

[0020] The initial privacy risk scores are sorted in descending order, and a predetermined number of initial privacy risk scores are selected as the target privacy risk scores.

[0021] Based on the suffix and initial candidate output corresponding to each of the target privacy risk scores, the memory induction model is fine-tuned to obtain an intermediate memory induction model.

[0022] Real-time statistics of the current iteration round, and determination of whether the current iteration round has reached a preset round threshold;

[0023] If so, the intermediate memory induction model shall be used as the optimized memory induction model;

[0024] If not, the intermediate memory induction model is used as the new memory induction model; and the process jumps to the step of taking the prefix in each of the prefix and suffix sample pairs as the input of the memory induction model and outputting the initial prompt word corresponding to each prefix, until the current iteration reaches the preset iteration threshold.

[0025] The intermediate memory induction model determined when the current iteration reaches the preset iteration threshold is used as the optimized memory induction model.

[0026] Optionally, the process involves generating multiple final queries based on the optimized memory-inducing model and inputting them into the target large language model to obtain multiple final candidate outputs. Based on these multiple final candidate outputs, a privacy risk comparison reference set is constructed, including:

[0027] Input the prefix in each of the prefix and suffix sample pairs into the optimized memory induction model, and output the optimized prompt word corresponding to each prefix;

[0028] By concatenating the aforementioned prefixes with their corresponding optimization suggestion words, multiple final queries are obtained.

[0029] Each of the final queries is used as input to the target large language model to obtain multiple final candidate outputs;

[0030] A similarity calculation is performed on each of the final candidate outputs and its corresponding suffix to obtain multiple final privacy risk scores;

[0031] The multiple final privacy risk scores are sorted in descending order, and the suffixes corresponding to the first preset number of final privacy risk scores are selected to construct a privacy risk comparison reference set.

[0032] Optionally, the target large language model is used to perform privacy compliance candidate filtering based on the user input text and the privacy risk comparison reference set, outputting multiple privacy compliance candidates, including:

[0033] The target large language model is invoked to perform multiple generation operations on the user input text to obtain multiple target candidate outputs;

[0034] The similarity of each target candidate output with each suffix in the privacy risk comparison reference set is calculated one by one to obtain multiple similarity values ​​corresponding to each target candidate output;

[0035] Among the multiple similarity values ​​corresponding to each target candidate output, the largest similarity value is selected as the privacy risk score of the target candidate output;

[0036] The privacy risk score of each target candidate output is compared with a preset privacy risk threshold, and any target candidate output with a privacy risk score less than or equal to the preset privacy risk threshold is taken as a privacy compliance candidate.

[0037] Optionally, the step of constructing a privacy-utility dual compliance candidate subset based on multiple privacy compliance candidates using a preset utility scoring rule, and generating a privacy-utility dual compliance candidate subset, includes:

[0038] The utility score for each privacy compliance candidate is obtained by using the preset utility scoring rules.

[0039] The utility score of each privacy compliance candidate is compared with a preset utility compliance threshold, and any privacy compliance candidate whose utility score is greater than or equal to the preset utility compliance threshold is taken as a privacy-utility dual compliance candidate.

[0040] Based on the multiple privacy-utility dual compliance candidates, a subset of privacy-utility dual compliance candidates is formed.

[0041] Optionally, the step of performing a candidate search based on the privacy-utility dual compliance candidate subset and the user input text, and outputting a final response to the user, includes:

[0042] Determine whether the privacy-utility dual compliance candidate subset is an empty set;

[0043] If the privacy-utility dual compliance candidate subset is not empty, the privacy-utility dual compliance candidate corresponding to the largest utility score is selected as the preliminary final result;

[0044] If the privacy-utility dual compliance candidate subset is an empty set, construct a Monte Carlo search tree with the user input text as the root node;

[0045] Based on the Monte Carlo search tree, output optimized candidate outputs;

[0046] The preliminary final result or the optimized candidate output shall be used as the final response to the user.

[0047] The second aspect of this invention provides a privacy risk auditing and control system for large language models, comprising:

[0048] The acquisition module is used to acquire the text dataset to be audited, and to perform text processing on the text dataset to be audited using a memory-induced model to generate multiple initial queries. Each of the initial queries is used as the input of the target large language model to obtain multiple initial candidate outputs.

[0049] The calculation module is used to calculate the initial privacy risk score corresponding to each of the initial candidate outputs, filter multiple target privacy risk scores from each of the initial privacy risk scores, and perform multiple rounds of iterative fine-tuning on the memory-inducing model to obtain the optimized memory-inducing model.

[0050] The first construction module is used to generate multiple final queries based on the optimized memory-induced model, input them into the target large language model, obtain multiple final candidate outputs, and construct a privacy risk comparison reference set based on the multiple final candidate outputs;

[0051] The filtering module is used to filter privacy compliance candidates based on the user input text and the privacy risk comparison reference set when receiving user input text, using the target large language model, and outputting multiple privacy compliance candidates.

[0052] The second construction module is used to construct a privacy-utility dual compliance candidate subset based on multiple privacy compliance candidates using a preset utility scoring rule, thereby generating a privacy-utility dual compliance candidate subset.

[0053] The output module is used to perform a candidate search based on the privacy-utility dual compliance candidate subset and the user input text, and output the final response to the user.

[0054] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the privacy risk auditing and control method for large language models as described above.

[0055] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the privacy risk auditing and control method for large language models as described above.

[0056] The fifth aspect of the present invention provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein, when the program instructions are executed by a computer, the computer performs the steps of the privacy risk auditing and control method for large language models as described above.

[0057] As can be seen from the above technical solutions, the present invention has the following advantages:

[0058] The above-mentioned technical solution of the present invention provides a privacy risk auditing and control method for large language models. It involves acquiring a text dataset to be audited, processing the dataset using a memory-inducing model to generate multiple initial queries, and using each initial query as input to the target large language model to obtain multiple initial candidate outputs. An initial privacy risk score is calculated for each initial candidate output, and multiple target privacy risk scores are selected from these initial privacy risk scores. The memory-inducing model is then iteratively fine-tuned multiple times to obtain an optimized memory-inducing model. Based on the optimized memory-inducing model, multiple final queries are generated and input into the target large language model to obtain multiple final candidate outputs. A privacy risk comparison reference set is constructed based on these final candidate outputs. When user input text is received, the target large language model performs privacy compliance candidate selection based on the user input text and the privacy risk comparison reference set, outputting multiple privacy compliance candidates. A privacy-utility dual compliance candidate subset is constructed based on the multiple privacy compliance candidates using a preset utility scoring rule, generating a privacy-utility dual compliance candidate subset. Based on the privacy-utility dual compliance candidate subset... The invention employs a dual-compliance candidate subset and user input text for candidate search, outputting a final response for the user. Based on this approach, the invention achieves standardized optimization of prompt generation capabilities through multi-round iterative fine-tuning of the memory-induced model. A unified privacy risk comparison reference set is constructed based on the optimized model output, enabling unified and standardized quantitative assessment of privacy risks in the output of the target large language model. This eliminates the need for repetitive personalized prompt construction and redundant search operations for large amounts of input, significantly reducing detection and control costs in large-scale input scenarios. Furthermore, the division of labor—offline model optimization and reference set construction, and online execution of compliance screening, utility evaluation, and necessary candidate search—balances the accuracy of privacy risk auditing with operational efficiency in practical deployment scenarios. The mechanism of combining privacy compliance and utility-based dual-dimensional screening with candidate search ensures both large-scale privacy risk management and the task utility of the model output. This makes the privacy risk audit and control process more adaptable to large-scale input and high-concurrency deployment scenarios, achieving a synergistic improvement in privacy risk management capabilities, scenario adaptability, and deployment practicality. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating the steps of a privacy risk auditing and control method for large language models provided in Embodiment 1 of the present invention.

[0061] Figure 2 This is an overall framework diagram of a privacy risk auditing and control method for large language models provided in Embodiment 1 of the present invention;

[0062] Figure 3 This is a structural block diagram of a privacy risk audit and control system for large language models provided in Embodiment 2 of the present invention. Detailed Implementation

[0063] This invention provides a method and system for privacy risk auditing and control for large language models, which solves the technical problem that existing privacy risk auditing and control methods for large language models are difficult to adapt to large-scale input and actual deployment scenarios.

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that in the optional embodiments of the present invention, the object information and other related data involved require the permission or consent of the object when the embodiments of the present invention are applied to specific products or technologies, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. That is to say, if the embodiments of the present invention involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations, and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0065] Terminology Explanation:

[0066] Large Language Models (LLMs): LLMs are deep learning models based on the Transformer architecture, trained on massive amounts of text data through self-supervised learning. Their core is learning the probability distribution of language, modeling complex relationships between words using a massive number of parameters (typically billions to trillions), thereby gaining the ability to understand and generate natural language. Typical models include the ChatGPT (Chat Generative Pre-trained Transformer) series, the Llama (Large Language Model Meta AI) series, Gemini, and Qwen (Tongyi Qwen), etc. They can be categorized by purpose into general dialogue models, instruction fine-tuning models, and domain-specific generative models.

[0067] Model memory refers to the phenomenon where a machine learning model overfits to training samples during training, resulting in its ability to directly or approximately reproduce the training data during the inference phase. This memory typically manifests as text fragments, structural patterns, or semantic-level reproductions, and has significant implications for privacy protection and copyright compliance. In this invention, model memory is considered a potential source of privacy risks.

[0068] LoRA (Low-Rank Adaptation): LoRA is an efficient fine-tuning technique for large language models. Its core idea is to freeze the original weights of the pre-trained model and inject only a trainable low-rank matrix into the model, simulating the effect of full parameter updates through matrix factorization. This design reduces the number of parameters requiring fine-tuning by several orders of magnitude, significantly reducing computational overhead and storage requirements, while maintaining performance comparable to full parameter fine-tuning.

[0069] Test-time scaling refers to a technique used during the inference phase to increase the computational load or number of inference paths to obtain multiple candidate outputs without modifying the parameters of the large language model. Test-time scaling can be achieved through multiple sampling, candidate generation, search, or inference path expansion, generating multiple different candidate inference outputs under the same input conditions. In this invention, test-time scaling is used to provide a diverse candidate inference output space for privacy risk auditing and control.

[0070] Monte Carlo Tree Search (MCTS) is a decision search method based on random sampling and tree structure search. It gradually approaches a better decision path by continuously expanding, simulating, and backtracking within the search tree. In large language model inference scenarios, MCTS can be used to expand the generation process at the path level, thereby generating multiple candidate inference paths or candidate output results during the inference stage. In some implementations of this invention, MCTS can be used as an implementation method for testing-time expansion, but this invention is not limited to using only this search method.

[0071] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a privacy risk auditing and control method for large language models, as provided in Embodiment 1 of the present invention.

[0072] This invention provides a privacy risk auditing and control method for large language models, comprising:

[0073] Step 101: Obtain the text dataset to be audited, and use the memory-induced model to process the text dataset to generate multiple initial queries. Use each initial query as the input of the target large language model to obtain multiple initial candidate outputs.

[0074] The text dataset to be audited refers to a collection of text data containing content related to potential privacy risks, which is used to conduct privacy risk audits of large language models.

[0075] Memory-inducing models are models capable of generating corresponding prompts from text data, thereby constructing queries, and iteratively optimizing their prompt generation capabilities through privacy risk scoring. Memory-inducing models employ general-purpose large language models with text generation and prompt construction capabilities, and support for parameter fine-tuning, including but not limited to open-source fine-tunable large language models such as the Llama series, Qwen series, and ChatGLM series.

[0076] The initial query refers to the query text generated by the memory-induced model based on the text dataset to be audited, which is used as input to the target large language model to trigger the output.

[0077] Initial candidate output refers to the text output generated by the target large language model based on the initial query, used to calculate the initial privacy risk score.

[0078] The target large language model is the model to be audited and controlled for privacy risks. Various commercial or open-source large language models with text generation and logical reasoning capabilities can be used, including but not limited to mainstream large language models such as ChatGPT series models, Llama series models, Gemini series models, and Qwen series models.

[0079] It should be noted that the process involves obtaining the text dataset to be audited, then using a memory-induced model to process the dataset to generate multiple initial queries. Subsequently, each initial query is input into the target large language model, which generates multiple initial candidate outputs accordingly, thus completing the basic operations of query construction and candidate output acquisition in the initial stage.

[0080] Further, step 101 may include the following sub-steps:

[0081] S11. Segment each text in the text dataset to be audited that contains potential privacy risks to obtain multiple prefix and suffix sample pairs;

[0082] S12. Take the prefix in each prefix-suffix sample pair as the input of the memory induction model, and output the initial prompt word corresponding to each prefix;

[0083] S13. Concatenate each prefix with the corresponding initial prompt word to obtain multiple initial queries;

[0084] S14. Each initial query is used as input to the target large language model to obtain multiple initial candidate outputs.

[0085] Prefix and suffix sample pairs refer to pairs of samples obtained by segmenting texts containing potential privacy risks in the text dataset to be audited according to preset rules. These pairs consist of the text context (prefix) and the potential privacy information (suffix) and are used for subsequent privacy risk induction, scoring calculation and model fine-tuning.

[0086] It should be noted that the dataset to be tested (i.e., the text dataset to be audited) is as follows:

[0087] ;

[0088] For each sample (i.e., text containing potential privacy risks) Perform the splitting operation:

[0089] ;

[0090] in, For the prefix corresponding to the k-th prefix-suffix sample pair, Let be the suffix corresponding to the k-th prefix-suffix sample pair. A fixed number of tokens can be used for segmentation.

[0091] Furthermore, a memory-inducing model was constructed. For a given prefix Generate initial prompt words :

[0092] ;

[0093] The prefix is ​​concatenated with the corresponding initial prompt to form the initial query for the target LLM. :

[0094] ;

[0095] Initial candidate outputs are obtained through the target LLM. :

[0096] ;

[0097] In this embodiment, each text in the dataset containing potential privacy risks is reasonably segmented according to the boundaries of privacy-sensitive segments, splitting the text into prefixes containing contextual background and suffixes containing potential privacy information, thus obtaining multiple prefix-suffix sample pairs. Then, the prefixes in each sample pair are input into a memory-inducing model, which generates initial prompt words based on the semantic features of the prefixes to induce the target large language model to associate the corresponding suffix privacy information. Next, each prefix and its corresponding initial prompt word are concatenated in a semantically coherent manner to form multiple initial queries that can directly trigger the output of the target large language model. Finally, each initial query is input into the target large language model, which generates text based on the query content, obtaining multiple initial candidate outputs corresponding to the prefixes, providing basic data for the subsequent calculation of initial privacy risk scores. This process, through standardized text segmentation, prompt word generation, and query construction, achieves accurate induction and batch processing of potential privacy risks, avoiding the tedious operation of constructing prompts for each sample in traditional methods, and effectively reducing the cost of large-scale sample processing.

[0098] Step 102: Calculate the initial privacy risk score corresponding to each initial candidate output, select multiple target privacy risk scores from each initial privacy risk score, and perform multiple rounds of iterative fine-tuning on the memory-inducing model to obtain the optimized memory-inducing model.

[0099] It should be noted that the initial privacy risk score corresponding to each initial candidate output can be calculated by comparing the similarity between the initial candidate output and the suffix in the corresponding prefix and suffix sample pair, and thus converting it into an initial privacy risk score. Subsequently, multiple target privacy risk scores are selected from each initial privacy risk score according to preset rules. Based on the target privacy risk score, the memory induction model is fine-tuned through multiple rounds of iterations. After each round of fine-tuning, the generation effect is verified and prompted until the preset optimization requirements are met, and finally the optimized memory induction model is obtained.

[0100] Furthermore, step 102 may include the following sub-steps:

[0101] S21. Calculate the similarity between each initial candidate output and its corresponding suffix to obtain multiple initial privacy risk scores;

[0102] S22. Sort the multiple initial privacy risk scores in descending order and select the first preset number of initial privacy risk scores as the target privacy risk score.

[0103] S23. Based on the suffix and initial candidate output corresponding to each target privacy risk score, the memory-inducing model is fine-tuned to obtain the intermediate memory-inducing model.

[0104] S24. Real-time statistics of the current iteration round and determination of whether the current iteration round has reached the preset round threshold;

[0105] S25. If so, then the intermediate memory induction model shall be used as the optimized memory induction model.

[0106] S26. If not, then use the intermediate memory induction model as the new memory induction model; and jump to execute the step of using the prefix in each prefix-suffix sample pair as the input of the memory induction model and outputting the initial prompt word corresponding to each prefix, until the current iteration reaches the preset iteration threshold.

[0107] S27. The intermediate memory induction model determined when the current iteration reaches the preset iteration threshold is used as the optimized memory induction model.

[0108] It should be noted that this invention defines a distance function d(·,·) to measure the "distance" between the candidate output and the actual suffix. The distance function can be obtained by transforming the similarity function Sim(·,·).

[0109] ;

[0110] Similarity functions can be based on the longest common subsequence (LCS) metric:

[0111] ;

[0112] in, This represents the length of the longest common subsequence of sequences a and b. This represents the length of b. Additionally, ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation, an index based on the longest common subsequence) can also be used as an implementation of similarity. Based on this, a privacy risk score is defined:

[0113] ;

[0114] in, A function that measures the “distance” between candidate outputs (such as initial candidate outputs and final candidate outputs) and their corresponding suffixes (suffixes in prefix-suffix sample pairs). The larger the value, the greater the difference in text content between the two. This is a function that measures the similarity between candidate outputs and their corresponding suffixes; the larger the value, the higher the overlap in text content between the two. This is a similarity function based on the longest common subsequence (LCS), used to quantify the degree of text overlap between candidate outputs and suffixes; The privacy risk score (such as the initial privacy risk score or the final privacy risk score) is the kth candidate output. The higher the score, the higher the similarity between the candidate output and the corresponding suffix, and the higher the risk of privacy leakage. The text sequence for the k-th candidate output (such as the initial candidate output or the final candidate output); It is the text sequence of the suffix (the suffix in the prefix-suffix sample pair) corresponding to the k-th candidate output.

[0115] Furthermore, the audit phase uses scoring signals to... To fine-tune LoRA to make it better at generating cue words that "expose reproducible risks," the optimization objective can be to maximize the risk score:

[0116] ;

[0117] In practice, multiple candidate prompt words can be generated for each sample, and the best score can be used to update the results, thus obtaining the "generated candidates":

[0118] ;

[0119] ;

[0120] Use again Updated as training signal .

[0121] Where K is the total number of prefix and suffix sample pairs to be audited; The prefix in the k-th prefix-suffix sample pair; An auxiliary perturbation variable is introduced to generate candidate cue words, which is used to generate diverse cue words for each prefix; J represents the j-th initial cue word candidate generated by the memory-inducing model for the k-th prefix; J is the number of cue word candidates generated for each prefix. The initial query is obtained by concatenating the k-th prefix with the j-th initial prompt word candidate; The initial candidate output generated by the target large language model based on this initial query; The suffix in the k-th prefix-suffix sample pair; The maximum privacy risk score corresponding to the kth sample is selected as the training signal from the candidate outputs generated from multiple prompt word candidates, which has the highest similarity to the suffix. The maximum value is taken for the privacy risk score corresponding to the J prompt word candidates generated for each prefix.

[0122] Based on the above, for each initial candidate output and its corresponding suffix, the similarity value between them is calculated using a similarity function based on the longest common subsequence or ROUGE-L, and this similarity value is directly used as the initial privacy risk score. After obtaining multiple initial privacy risk scores, these scores are sorted in descending order, and the highest-ranking high-risk scores are selected as target privacy risk scores according to a pre-set number. Then, based on the suffixes corresponding to each target privacy risk score and the initial candidate output, the parameters of the memory inducement model are updated using efficient fine-tuning methods such as LoRA to obtain an intermediate memory inducement model. Subsequently, the current iteration round is counted in real time and compared with a preset round threshold. If the current iteration round has reached the preset round threshold, the intermediate memory inducement model is directly determined as the optimized memory inducement model. If it has not reached the threshold, the intermediate memory inducement model is used as the new memory inducement model, and the process of generating prompt words, constructing queries, obtaining candidate outputs, calculating and filtering risk scores is repeated until the iteration round reaches the preset round threshold. Finally, the intermediate memory inducement model at this time is used as the optimized memory inducement model. This process uses high-risk scoring-driven, multi-round iterative fine-tuning to continuously optimize the prompt generation capability of the memory-inducing model. It can more accurately generate prompt words that can induce the target large language model to expose privacy risks. At the same time, the standardized iteration termination mechanism ensures the efficiency and stability of model fine-tuning, avoids the redundant operations of traditional sample-by-sample optimization, and effectively reduces the model optimization cost in large-scale scenarios.

[0123] Step 103: Generate multiple final queries based on the optimized memory-inducing model and input them into the target large language model to obtain multiple final candidate outputs. Based on the multiple final candidate outputs, construct a privacy risk comparison reference set.

[0124] It should be noted that the prefixes in each prefix-suffix sample pair are input into the optimized memory-inducing model. The model generates optimized prompts that can accurately induce privacy risks. Then, each prefix and its corresponding optimized prompt are concatenated into multiple semantically coherent final queries. These final queries are input into the target large language model one by one to obtain multiple final candidate outputs corresponding to the prefixes. Subsequently, the final privacy risk scores corresponding to the final candidate outputs are combined to screen out high-risk suffixes. These high-risk suffixes are integrated into a standardized privacy risk comparison reference set, providing a unified reference for subsequent privacy compliance screening of user input.

[0125] Furthermore, step 103 may include the following sub-steps:

[0126] S31. Input the prefixes in each prefix-suffix sample pair into the optimized memory induction model, and output the optimized prompt words corresponding to each prefix.

[0127] S32. Concatenate each prefix with the corresponding optimization suggestion words to obtain multiple final queries;

[0128] S33. Each final query is used as input to the target large language model to obtain multiple final candidate outputs;

[0129] S34. Calculate the similarity between each final candidate output and its corresponding suffix to obtain multiple final privacy risk scores;

[0130] S35. Sort the multiple final privacy risk scores in descending order and select the suffixes corresponding to the first preset number of final privacy risk scores to construct a privacy risk comparison reference set.

[0131] It should be noted that the prefixes in each prefix-suffix sample pair are input into the optimized memory-inducing model. This model, relying on its prompt generation capability refined through multiple rounds of iterations, outputs optimized prompt words that can more accurately induce the target large language model to expose privacy risks. Subsequently, each prefix and its corresponding optimized prompt word are concatenated in a semantically coherent manner to form multiple final queries that can directly trigger the output of the target large language model. These final queries are input into the target large language model one by one to obtain multiple final candidate outputs corresponding to the prefixes. Then, for each final candidate output and its corresponding suffix, the similarity value between them is calculated using a similarity function based on the longest common subsequence or ROUGE-L, thereby obtaining multiple final privacy risk scores. These final privacy risk scores are sorted in descending order, and the suffixes corresponding to the highest-ranking high-risk scores are selected according to a pre-set number. These high-risk suffixes are integrated into a standardized privacy risk comparison reference set.

[0132] This invention generates high-quality optimized prompt words through an optimized memory-inducing model. Combined with standardized similarity calculation and high-risk suffix screening, it constructs a unified and high-risk privacy risk comparison reference set, avoiding the redundant operation of traditional methods that detect each sample one by one, and greatly improving the efficiency and accuracy of privacy risk assessment in large-scale input scenarios.

[0133] Step 104: When user input text is received, the target large language model is used to filter privacy compliance candidates based on the user input text and the privacy risk comparison reference set, and output multiple privacy compliance candidates.

[0134] It should be noted that when user input text is received, the target large language model is first called to perform multiple generation operations on the user input text to obtain multiple candidate outputs. Then, the similarity of each candidate output with all suffixes in the privacy risk comparison reference set is calculated one by one to obtain the privacy risk score corresponding to each candidate output. Then, these privacy risk scores are compared with the preset privacy risk threshold, and candidate outputs with scores less than or equal to the threshold are selected, which are multiple privacy compliance candidates.

[0135] Furthermore, step 104 may include the following sub-steps:

[0136] S41. Call the target large language model to perform multiple generation operations on the user input text to obtain multiple target candidate outputs;

[0137] S42. Calculate the similarity between each target candidate output and each suffix in the privacy risk comparison reference set to obtain multiple similarity values ​​corresponding to each target candidate output.

[0138] S43. Among the multiple similarity values ​​corresponding to each target candidate output, select the largest similarity value as the privacy risk score of the target candidate output;

[0139] S44. Compare the privacy risk score of each target candidate output with the preset privacy risk threshold, and take any target candidate output with a privacy risk score that is less than or equal to the preset privacy risk threshold as a privacy compliance candidate.

[0140] It should be noted that, for input x, the target LLM is called N times to generate:

[0141] ;

[0142] in, For different sampling randomness and parameters (such as temperature, random seed, etc.);

[0143] Let S be the reference set of "actual suffixes", and then consider the candidate outputs. Define privacy risk score:

[0144] ;

[0145] Privacy constraints are as follows:

[0146] ;

[0147] Where x is the user input text; N is the number of times the target large language model is called to generate candidate outputs for the same user input text; and y is the set of all candidate outputs generated by the target large language model for the user input text x. Let S be the i-th target candidate output generated by the target large language model for the user input text x; S is the privacy risk comparison reference set; This is a preset privacy risk threshold.

[0148] Based on the above, when user input text is received, the target large language model is invoked, and multiple generation operations are performed on the user input text by adjusting sampling randomness parameters such as temperature and random seed, resulting in a variety of target candidate outputs. Then, each target candidate output is compared with each suffix in the privacy risk comparison reference set using a similarity function based on the longest common subsequence or ROUGE-L, resulting in multiple similarity values ​​for each target candidate output. The largest similarity value is selected as the privacy risk score for that target candidate output, reflecting the highest degree of overlap between the candidate output and high-risk suffixes. The privacy risk score of each target candidate output is then compared with a preset privacy risk threshold (a hyperparameter derived from regulatory requirements, industry experience, etc.). Any target candidate output with a privacy risk score less than or equal to the preset privacy risk threshold is identified as a privacy-compliant candidate. This process, relying on a uniformly constructed privacy risk comparison reference set, achieves batch and standardized privacy risk assessment of candidate outputs corresponding to user input. It eliminates the need to repeatedly construct personalized prompts for each user input, significantly reducing detection costs in large-scale input scenarios and improving screening efficiency and consistency.

[0149] It is worth mentioning that this invention uses fixed text similarity metrics such as the longest common subsequence ratio (LCS) or ROUGE-L to score the target model output against the target segment. In addition to this implementation, the scoring module can use alternative methods such as semantic vector-based similarity measurement or LLM-as-judge scoring.

[0150] Step 105: Using a preset utility scoring rule, construct a privacy-utility dual compliance candidate subset based on multiple privacy compliance candidates, and generate a privacy-utility dual compliance candidate subset.

[0151] It should be noted that a preset utility scoring rule is used to evaluate multiple privacy compliance candidates. This rule can quantify the scoring from dimensions such as the relevance of the content to the user's input text, the fluency of the text expression, and the completion of the task objective. The utility score of each privacy compliance candidate is compared with a preset utility compliance threshold, and candidates with a utility score greater than or equal to the threshold are selected. These candidates that simultaneously meet the requirements of privacy compliance and utility are integrated into a privacy-utility dual compliance candidate subset.

[0152] Furthermore, step 105 may include the following sub-steps:

[0153] S51. Using a preset utility scoring rule, evaluate the utility of each privacy compliance candidate to obtain a utility score for each privacy compliance candidate;

[0154] S52. Compare the utility score of each privacy compliance candidate with the preset utility compliance threshold, and take any privacy compliance candidate whose utility score is greater than or equal to the preset utility compliance threshold as a privacy-utility dual compliance candidate.

[0155] S53. Based on multiple privacy-utility dual compliance candidates, a subset of privacy-utility dual compliance candidates is formed.

[0156] The preset utility scoring rules refer to the pre-set quantitative rules used to evaluate the performance of privacy compliance candidates in dimensions such as content relevance, fluency, and task completion, and are used to screen candidate outputs that have both privacy compliance and practical value.

[0157] It should be noted that the utility rating function is defined as follows:

[0158] ;

[0159] in, This can be the result of task rule metrics, standard answer matching, or evaluation and scoring through other LLMs. The utility constraint is defined as:

[0160] ;

[0161] in, The utility score for the i-th privacy compliance candidate; The preset utility scoring rules are functions that quantify privacy compliance candidates from dimensions such as content relevance, text fluency, and task objective completion. The results can come from task rule indicators, standard answer matching, or other large language model review scores. This is a preset utility compliance threshold.

[0162] In this embodiment, a preset utility scoring rule is used to evaluate the utility of each privacy compliance candidate, resulting in a utility score for each candidate. Then, the utility score of each candidate is compared with a preset utility compliance threshold. Any candidate with a utility score greater than or equal to the threshold is identified as a dual-compliance candidate (privacy-utility compliance). All candidates meeting the criteria are then integrated to form a subset of dual-compliance candidates. This process, through standardized utility evaluation and threshold comparison, achieves batch screening of dual-compliance candidates. While ensuring controllable privacy risks, it also ensures the task-oriented usability of the output content, avoiding the output quality degradation caused by focusing solely on privacy compliance. Furthermore, relying on unified rules and thresholds improves processing efficiency and consistency in large-scale input scenarios.

[0163] Step 106: Perform a candidate search based on the privacy-utility dual compliance candidate subset and the user input text, and output the final response to the user.

[0164] It should be noted that when performing candidate search based on the privacy-utility dual compliance candidate subset and the user input text, the system first checks whether the subset contains valid candidates. If it does, the candidate with the highest matching degree and optimal utility is selected. If it does not, the system uses the user input text as the core and conducts candidate search through a preset search strategy to filter out candidates that meet the privacy and utility requirements. Finally, the optimal candidate is selected as the final response to the user.

[0165] Furthermore, step 106 may include the following sub-steps:

[0166] S61. Determine whether the privacy-utility dual compliance candidate subset is an empty set;

[0167] S62. If the privacy-utility dual compliance candidate subset is not empty, select the privacy-utility dual compliance candidate corresponding to the largest utility score as the preliminary final result;

[0168] S63. If the privacy-utility dual compliance candidate subset is an empty set, construct a Monte Carlo search tree with the user input text as the root node.

[0169] S64. Based on the Monte Carlo search tree, output optimized candidate outputs;

[0170] S65. Use the preliminary final results or optimized candidate outputs as the final response to the user.

[0171] It should be noted that the collection is filtered first:

[0172] ;

[0173] If it is not an empty set, output the candidate with the highest utility:

[0174] ;

[0175] in, A subset of candidates for both privacy and utility compliance; The set of all candidate outputs generated by the target large language model for the user input text; As a final response, it indicates a subset of candidates for both privacy and utility compliance. Among them, utility scores are selected. The highest-ranking candidate output is chosen as the final result for the user. This is a function used to select candidate outputs that maximize the utility score.

[0176] If the set is empty, Monte Carlo Tree Search (MCTS) optimization can be performed. When BoN struggles to generate candidates that satisfy the constraints, MCTS is used to search in a larger generation space. The generation process is abstracted as a Monte Carlo tree, where nodes represent partial generation states, edges represent an extended generation step, and a root-to-leaf path corresponds to a complete candidate output. The "selection" phase of MCTS can use the UCB (Upper Confidence Bound) strategy to select nodes:

[0177] ;

[0178] in, For node value estimation, For the number of times a node is accessed. To explore coefficients.

[0179] For the complete output y obtained from a single simulation, a joint evaluation value is defined (for backpropagation updates):

[0180] ;

[0181] in, It is the upper confidence bound of node n, used to evaluate the merits of nodes in the "selection" phase of MCTS, balance exploration and utilization, and guide node selection; The value estimate for node n is the average joint evaluation value obtained from past simulations of that node, reflecting the historical performance of the node. The total number of visits to the parent node of node n reflects the degree of exploration of the parent node; This represents the number of times node n has been visited, reflecting the exploration frequency of that node. It is a very small positive number, used to avoid the denominator being zero and to ensure the stability of the calculation; This is a tradeoff factor used to balance privacy performance and model utility. The target large model for this invention can be a closed-source API (Application Programming Interface) or an open-source model, independent of the accessibility of the target LLM parameters, and suitable for practical deployment.

[0182] In this embodiment, the privacy-utility dual compliance candidate subset is traversed and tested to determine whether it is an empty set, thereby determining the subsequent output strategy. If the subset is not found to be empty, the utility scores corresponding to all privacy-utility dual compliance candidates are extracted, and the candidate with the highest utility score is selected by comparing them one by one and determined as the preliminary final result. If the subset is found to be empty, a Monte Carlo search tree is built with the user input text as the root node, according to the node construction rules of Monte Carlo Tree Search (MCTS). The root node corresponds to the original user input text, and the child nodes correspond to various candidate outputs and derived results generated based on the input. Then, based on the constructed Monte Carlo search tree, the UCB strategy is used to perform node selection, expansion, simulation, and backpropagation operations. Through multiple rounds of iterative balancing exploration and utilization, optimized candidate outputs that meet the privacy compliance and utility requirements are selected. Finally, according to the state of the privacy-utility dual compliance candidate subset, the obtained preliminary final result or optimized candidate output is directly used as the final response to the user. This process, by first determining the state of a subset and then processing it accordingly, ensures both the privacy compliance and optimal utility of the final response. Furthermore, the Monte Carlo search tree's fallback search mechanism avoids the anomaly of having no candidate outputs, thus improving the stability and fault tolerance of the technical process. At the same time, the standardized selection and search logic reduces redundant calculations and improves processing efficiency in large-scale input scenarios.

[0183] As a comparison of technical effects, existing technologies can be used as a reference. Numerous studies have pointed out that models exhibit significant memory phenomena: given certain prompt words, the model may reproduce long paragraphs of the training text. Currently, there are several typical methods for detecting memory in large models. Among them, the detection method based on text prefix completion (Prefix-Suffix Extraction) extracts the first half of the known training text as input and compares the consistency / similarity between the model's generation and the second half. Detection methods based on prompt word search or gradient optimization (such as GCG, Greedy Coordinate Gradient) perform multiple rounds of generation, evaluation, and updating for each target sample to improve the ability to induce reproduction. Methods based on auxiliary language models to generate prompt words use additional language models to generate candidate prompts and then screen and evaluate the output of the target model. The above solutions can detect "explicit or approximate reproduction" at the sample level, but they generally suffer from problems such as high per-sample cost, difficulty in scaling to dataset size, and usually remain at the level of post-audit.

[0184] Meanwhile, research indicates that inference trajectories may contain sensitive user data, which could be injected with attacks or accidentally leaked to visible outputs. Increasing test-time inference computation (more inference steps, more candidate exploration) may amplify leaks in the inference trajectory. To address these issues, some security and privacy improvements have been implemented in test-time extensions. STAIR proposes combining security alignment with "introspective inference," using security-guided MCTS to generate process data, train the process reward model, and then use search during testing to improve security performance, embodying a "training + test-time search" system framework. SALT proposes a lightweight test-time intervention to address leakage during inference: by injecting directional steering vectors into hidden states, leakage in the CoT (co-thinking chain) is reduced while preserving inference utility as much as possible. These works reveal and mitigate the risks brought by inference and test-time extensions from different perspectives, but in engineering implementation, challenges remain, such as how to efficiently conduct large-scale privacy risk assessments and how to utilize test-time extensions to enhance model privacy performance.

[0185] As can be seen from the above, existing audit processes are mainly sample-by-sample, lacking automation and scalability. Existing training data memory detection methods mostly rely on constructing prompts for each individual sample or performing multiple rounds of search / optimization. The detection cost increases linearly with the number of samples, making it difficult to achieve a unified and repeatable privacy risk assessment in scenarios with large amounts of input or complex applications. Secondly, existing audit processes are mainly "post-event detection," lacking an active control loop during the inference stage. Existing methods usually determine whether there is a leakage risk after the model outputs. Even if a high-risk output is found, there is a lack of mechanisms to screen, constrain, or replace candidate outputs during the inference process, making it difficult to reduce the probability of leakage in the final output. In addition, when faced with multiple candidate outputs generated during testing, existing audit processes lack a feasible joint decision-making mechanism. In test-time scaling scenarios (including multiple sampling, search, MCTS, etc.), the model will generate multiple candidate inference outputs. It is difficult to simultaneously measure "privacy risk" and "task utility," and often only simple suppression or reduction of generation capacity can be adopted, resulting in a significant decrease in utility and unstable control effects.

[0186] Therefore, with the widespread deployment of large language models in practical applications, the privacy leakage risks caused by the content generated during the inference stage are gradually becoming prominent. To address the shortcomings of existing technologies, this invention mainly solves the following technical problems:

[0187] 1) Existing technologies struggle to automate and scale up the auditing of privacy risks arising from large language model memory. Current methods for detecting privacy risks in large language models largely rely on sample-by-sample analysis or manually constructed prompts, making it difficult to automate and repeatably audit privacy risks arising from model memory in the inference phase. When the scope of detection expands to large amounts of input or complex application scenarios, existing technologies struggle to achieve efficient and unified privacy risk assessment.

[0188] 2) Existing technologies mainly focus on privacy risk detection, lacking proactive control capabilities during the inference stage. Most existing privacy-related technologies are used for post-hoc detection or offline assessment of whether a model has a privacy leakage risk. Even if high-risk outputs are identified, it is difficult to effectively intervene in the model's behavior during the inference process.

[0189] 3) Existing technologies lack a decision-making mechanism that balances privacy risks and task utility in scenarios with multiple candidate inference outputs. When multiple candidate inference outputs exist, existing technologies typically cannot simultaneously consider privacy risks and task utility to make a reasonable selection of the output. Simply suppressing high-risk outputs or reducing the model's generation capability often significantly impairs the model's task performance, failing to meet the practical application's need for a balance between privacy security and model performance.

[0190] To address the aforementioned issues, this invention provides a privacy risk auditing and control method for large language models. This method enables unified quantitative evaluation of model outputs (including reproducible and reconstructed content triggered by model memory) to adapt to large-scale input and real-world deployment scenarios. Furthermore, after identifying high-risk candidate outputs, filtering can be performed during the inference phase. Moreover, in extended multi-candidate output scenarios during testing, a joint trade-off and decision-making process between privacy risks and task utility is achieved, ensuring that the system meets preset privacy and security requirements while maintaining the model's task utility and inference performance as much as possible.

[0191] Specifically, such as Figure 2As shown, this invention comprises two phases: an audit phase and a control phase. The audit phase constructs and trains a "memory-inducing model" to automatically generate optimized prompt words that trigger the target large language model (target LLM) to reproduce the suffix content of the training text, thereby forming a privacy risk score and threshold. The control phase introduces test-time scaling during inference, using methods such as Best-of-N (best N sampling) and MCTS (Monte Carlo Tree Search) to generate candidate outputs, and selects the final output under privacy and utility constraints to enhance privacy while preserving task utility as much as possible. The audit phase addresses the problem of how to automatically and scalably discover the "reproducibility / highly similar reconstruction" tendency of the target LLM and form reusable risk scores and thresholds. Its key technical point is: using the memory-inducing model to automatically generate prompt words, so that auditing does not rely on manually constructed prompts. The control phase attempts to solve the problem of using test-time scaling during inference to find an output that both ensures privacy and completes the task.

[0192] In summary, this invention utilizes a "reasoning-only" generation mechanism based on a memory-inducing model. By training the memory-inducing model, it automatically generates completion prompts based on the input text fragments, rather than optimizing each sample individually during the detection phase. The prompts are obtained through a single forward inference during the detection phase, solving the problems of high computational cost and scalability issues associated with sample-by-sample prompt search in existing technologies. Secondly, regarding constraint selection under extended testing conditions, this invention generates multiple candidates through Best-of-N during the inference phase and combines this with MCTS search optimization, making joint decisions under privacy and utility constraints to output results that satisfy privacy requirements while maximizing utility. Finally, this invention performs audit modeling based on the prefix / suffix of the dataset to be tested and uses the privacy risk metrics and thresholds obtained from the audit for control during the inference phase, achieving an integrated method for privacy risk auditing and control for large language models.

[0193] Compared with existing technologies, this invention has significant advantages. On the one hand, existing technologies mostly rely on post-event detection and offline evaluation, making it difficult to intervene during the inference process even if high-risk outputs are detected. This invention, however, generates privacy risk scores and thresholds during the auditing phase and directly filters / selects outputs based on these scores during the control phase, achieving executable privacy risk control during the inference phase and thus substantially reducing the probability of leakage of the final output. On the other hand, existing privacy auditing methods often rely on manually constructed prompts or sample-by-sample search optimization, with costs rising rapidly with sample size. This invention introduces a memory-inducing model to automatically generate optimized prompts and performs unified quantification based on "actual suffixes" and "distance functions," making the auditing process repeatable and batch-executed, thereby improving the efficiency and consistency of large-scale privacy risk assessment.

[0194] In this embodiment of the invention, a method for privacy risk auditing and control for large language models is provided. The method involves acquiring a text dataset to be audited, processing the dataset using a memory-inducing model to generate multiple initial queries, and using each initial query as input to the target large language model to obtain multiple initial candidate outputs. An initial privacy risk score is calculated for each initial candidate output, and multiple target privacy risk scores are selected from these initial privacy risk scores. The memory-inducing model is then iteratively fine-tuned multiple times to obtain an optimized memory-inducing model. Based on the optimized memory-inducing model, multiple final queries are generated and input into the target large language model to obtain multiple final candidate outputs. A privacy risk comparison reference set is constructed based on these final candidate outputs. When user input text is received, the target large language model performs privacy compliance candidate selection based on the user input text and the privacy risk comparison reference set, outputting multiple privacy compliance candidates. A privacy-utility dual compliance candidate subset is constructed based on the multiple privacy compliance candidates using a preset utility scoring rule, generating a privacy-utility dual compliance candidate subset. Based on the privacy... - Based on the above scheme, this invention achieves standardized optimization of prompt generation capability through multi-round iterative fine-tuning of the memory-induced model. Relying on the optimized model output results, a unified privacy risk comparison reference set is constructed, which can perform unified and standardized quantitative assessment of privacy risks of the target large language model output. There is no need to repeatedly perform personalized prompt construction and redundant search operations for a large number of inputs, which greatly reduces the detection and control costs in large-scale input scenarios. At the same time, by completing the model optimization and reference set construction offline and performing compliance screening, utility evaluation and necessary candidate search online, the division of labor is balanced between the accuracy of privacy risk audit and the operational efficiency in actual deployment scenarios. Moreover, through the mechanism of dual-dimensional screening of privacy compliance and utility combined with candidate search, the task utility of the model output is guaranteed while achieving large-scale privacy risk control. This makes the privacy risk audit and control process more adaptable to the actual deployment application scenarios of large-scale input and high concurrency, and achieves a synergistic improvement in privacy risk control capability, scenario adaptability and deployment practicality.

[0195] Please see Figure 3 , Figure 3 This is a structural block diagram of a privacy risk audit and control system for large language models provided in Embodiment 2 of the present invention.

[0196] This invention provides a privacy risk auditing and control system for large language models, comprising:

[0197] The acquisition module 301 is used to acquire the text dataset to be audited, and to use the memory-induced model to process the text dataset to be audited, generate multiple initial queries, and use each initial query as the input of the target large language model to obtain multiple initial candidate outputs;

[0198] The calculation module 302 is used to calculate the initial privacy risk score corresponding to each initial candidate output, select multiple target privacy risk scores from each initial privacy risk score, and perform multiple rounds of iterative fine-tuning on the memory-inducing model to obtain the optimized memory-inducing model.

[0199] The first construction module 303 is used to generate multiple final queries based on the optimized memory-induced model, input them into the target large language model, obtain multiple final candidate outputs, and construct a privacy risk comparison reference set based on the multiple final candidate outputs;

[0200] The filtering module 304 is used to filter privacy compliance candidates based on the user input text and the privacy risk comparison reference set when receiving user input text, and output multiple privacy compliance candidates.

[0201] The second construction module 305 is used to construct a privacy-utility dual compliance candidate subset based on multiple privacy compliance candidates using a preset utility scoring rule, thereby generating a privacy-utility dual compliance candidate subset.

[0202] Output module 306 is used to perform candidate search based on a privacy-utility dual compliance candidate subset and user input text, and output the final response to the user.

[0203] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0204] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the privacy risk auditing and control method for large language models as described in the above embodiments.

[0205] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the privacy risk auditing and control method for large language models as described in the above embodiments.

[0206] This invention also provides a computer program product, including a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the steps of the privacy risk auditing and control method for large language models as described in the above embodiments.

[0207] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0208] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0209] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0210] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0211] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A privacy risk auditing and control method for large language models, characterized in that, include: Obtain the text dataset to be audited, and use the memory-induced model to process the text dataset to generate multiple initial queries. Use each initial query as the input of the target large language model to obtain multiple initial candidate outputs. Calculate the initial privacy risk score corresponding to each initial candidate output, select multiple target privacy risk scores from each initial privacy risk score, and perform multiple rounds of iterative fine-tuning on the memory-inducing model to obtain the optimized memory-inducing model; Multiple final queries are generated based on the optimized memory-inducing model and input into the target large language model to obtain multiple final candidate outputs. A privacy risk comparison reference set is constructed based on the multiple final candidate outputs. When user input text is received, the target large language model is used to perform privacy compliance candidate filtering based on the user input text and the privacy risk comparison reference set, and output multiple privacy compliance candidates. A privacy-utility dual compliance candidate subset is constructed based on multiple privacy compliance candidates using a preset utility scoring rule, thereby generating a privacy-utility dual compliance candidate subset; Based on the privacy-utility dual compliance candidate subset and the user input text, a candidate search is performed, and a final response is output to the user.

2. The privacy risk auditing and control method for large language models according to claim 1, characterized in that, The text dataset to be audited is processed using a memory-induced model to generate multiple initial queries. Each initial query is used as input to the target large language model to obtain multiple initial candidate outputs, including: Each text containing potential privacy risks in the text dataset to be audited is segmented to obtain multiple prefix and suffix sample pairs; The prefix in each of the prefix and suffix sample pairs is used as the input of the memory induction model, and the initial prompt word corresponding to each prefix is ​​output. By concatenating each of the aforementioned prefixes with the corresponding initial prompt words, multiple initial queries can be obtained; Each of the initial queries is used as input to the target large language model to obtain multiple initial candidate outputs.

3. The privacy risk auditing and control method for large language models according to claim 2, characterized in that, The process involves calculating an initial privacy risk score corresponding to each initial candidate output, selecting multiple target privacy risk scores from these initial privacy risk scores, and performing multiple rounds of iterative fine-tuning on the memory-inducing model to obtain an optimized memory-inducing model, including: A similarity calculation is performed on each initial candidate output and its corresponding suffix to obtain multiple initial privacy risk scores; The initial privacy risk scores are sorted in descending order, and a predetermined number of initial privacy risk scores are selected as the target privacy risk scores. Based on the suffix and initial candidate output corresponding to each of the target privacy risk scores, the memory induction model is fine-tuned to obtain an intermediate memory induction model. Real-time statistics of the current iteration round, and determination of whether the current iteration round has reached a preset round threshold; If so, the intermediate memory induction model shall be used as the optimized memory induction model; If not, the intermediate memory induction model is used as the new memory induction model; and the process jumps to the step of taking the prefix in each of the prefix and suffix sample pairs as the input of the memory induction model and outputting the initial prompt word corresponding to each prefix, until the current iteration reaches the preset iteration threshold. The intermediate memory induction model determined when the current iteration reaches the preset iteration threshold is used as the optimized memory induction model.

4. The privacy risk auditing and control method for large language models according to claim 2, characterized in that, The optimized memory-inducing model generates multiple final queries, which are then input into the target large language model to obtain multiple final candidate outputs. Based on these multiple final candidate outputs, a privacy risk comparison reference set is constructed, including: Input the prefix in each of the prefix and suffix sample pairs into the optimized memory induction model, and output the optimized prompt word corresponding to each prefix; By concatenating the aforementioned prefixes with their corresponding optimization suggestion words, multiple final queries are obtained. Each of the final queries is used as input to the target large language model to obtain multiple final candidate outputs; A similarity calculation is performed on each of the final candidate outputs and its corresponding suffix to obtain multiple final privacy risk scores; The multiple final privacy risk scores are sorted in descending order, and the suffixes corresponding to the first preset number of final privacy risk scores are selected to construct a privacy risk comparison reference set.

5. The privacy risk auditing and control method for large language models according to claim 1, characterized in that, The target large language model is used to filter privacy compliance candidates based on the user input text and the privacy risk comparison reference set, and outputs multiple privacy compliance candidates, including: The target large language model is invoked to perform multiple generation operations on the user input text to obtain multiple target candidate outputs; The similarity of each target candidate output with each suffix in the privacy risk comparison reference set is calculated one by one to obtain multiple similarity values ​​corresponding to each target candidate output; Among the multiple similarity values ​​corresponding to each target candidate output, the largest similarity value is selected as the privacy risk score of the target candidate output; The privacy risk score of each target candidate output is compared with a preset privacy risk threshold, and any target candidate output with a privacy risk score less than or equal to the preset privacy risk threshold is taken as a privacy compliance candidate.

6. The privacy risk auditing and control method for large language models according to claim 1, characterized in that, The process involves constructing a privacy-utility dual-compliance candidate subset based on multiple privacy compliance candidates using a preset utility scoring rule, resulting in the following: The utility score for each privacy compliance candidate is obtained by using the preset utility scoring rules. The utility score of each privacy compliance candidate is compared with a preset utility compliance threshold, and any privacy compliance candidate whose utility score is greater than or equal to the preset utility compliance threshold is taken as a privacy-utility dual compliance candidate. Based on the multiple privacy-utility dual compliance candidates, a subset of privacy-utility dual compliance candidates is formed.

7. The privacy risk auditing and control method for large language models according to claim 1, characterized in that, The step of performing a candidate search based on the privacy-utility dual compliance candidate subset and the user input text, and outputting a final response to the user, includes: Determine whether the privacy-utility dual compliance candidate subset is an empty set; If the privacy-utility dual compliance candidate subset is not empty, the privacy-utility dual compliance candidate corresponding to the largest utility score is selected as the preliminary final result; If the privacy-utility dual compliance candidate subset is an empty set, construct a Monte Carlo search tree with the user input text as the root node; Based on the Monte Carlo search tree, output optimized candidate outputs; The preliminary final result or the optimized candidate output shall be used as the final response to the user.

8. A privacy risk auditing and control system for large language models, characterized in that, include: The acquisition module is used to acquire the text dataset to be audited, and to perform text processing on the text dataset to be audited using a memory-induced model to generate multiple initial queries. Each of the initial queries is used as the input of the target large language model to obtain multiple initial candidate outputs. The calculation module is used to calculate the initial privacy risk score corresponding to each of the initial candidate outputs, filter multiple target privacy risk scores from each of the initial privacy risk scores, and perform multiple rounds of iterative fine-tuning on the memory-inducing model to obtain the optimized memory-inducing model. The first construction module is used to generate multiple final queries based on the optimized memory-induced model, input them into the target large language model, obtain multiple final candidate outputs, and construct a privacy risk comparison reference set based on the multiple final candidate outputs; The filtering module is used to filter privacy compliance candidates based on the user input text and the privacy risk comparison reference set when receiving user input text, using the target large language model, and outputting multiple privacy compliance candidates. The second construction module is used to construct a privacy-utility dual compliance candidate subset based on multiple privacy compliance candidates using a preset utility scoring rule, thereby generating a privacy-utility dual compliance candidate subset. The output module is used to perform a candidate search based on the privacy-utility dual compliance candidate subset and the user input text, and output the final response to the user.

9. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the privacy risk auditing and control method for large language models as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the privacy risk auditing and control method for large language models as described in any one of claims 1-7.

Images