An extensible and accurate privacy-preserving large language model inference method and system

Abstract

The application relates to the fields of information security and large language model services, and discloses an extensible and accurate privacy protection large language model reasoning method and system, which is executed by a noise sampler, receives queries from one or more users, the queries contain general instructions and encrypted or labeled private attributes; for each group of queries, a candidate replacement attribute set of the private attributes corresponding to each query is obtained, a replacement attribute is sampled from the candidate replacement attribute set to generate a noise query, and the noise query and the corresponding original query are mixed to form a batch request; the batch request is sent to a black box large language model of a service provider, and a batch response returned is received; the response corresponding to the noise query is identified and discarded from the batch response, and is forwarded to the corresponding user. The application can provide a practical, extensible and plug-and-play privacy protection solution for a scene using a commercial black box LLM API without relying on the cooperation of the service provider and sacrificing the output quality.

Description

Technical Field

[0001] This invention relates to the fields of information security and large language model services, specifically to a scalable and accurate privacy-preserving large language model inference method and system. Background Technology

[0002] With the rapid development of large-scale language models such as ChatGPT, Claude, and Gemini, cloud-based online inference services have become core infrastructure for industries such as medical diagnosis, financial analysis, and legal consulting. However, user-input prompts often contain highly sensitive information such as patient medical records, investment portfolios, and trade secrets. If this information is leaked to model service providers or third-party attackers, it will cause serious privacy violations and business losses.

[0003] Existing privacy-preserving inference techniques mainly fall into three categories, but all have significant limitations. The first category is cue word cleansing, which erases personally identifiable information through named entity recognition. However, when sensitive information is crucial to inference, cleansing directly reduces the usability of the output; for example, removing specific patient symptoms in a medical context would render diagnostic suggestions untargetable. The second category is secure multi-party computation, which relies on cryptographic mechanisms to protect user data privacy and provides strong privacy guarantees. However, it comes with significant computational and communication overhead and requires major modifications to the model architecture and deployment process, making integration with existing black-box large language models a significant challenge. The third category is differential privacy techniques, which add calibration noise to the input to provide formal privacy guarantees. However, the introduction of noise creates an inherent contradiction between data usability and privacy protection, making it difficult to simultaneously meet the requirements of high-precision inference.

[0004] In summary, existing technologies lack practicality due to their failure to find a viable balance between privacy, usability, and complexity. Most frameworks require significant modifications to the LLM architecture or expose parameter information of proprietary models, severely hindering their deployment in real-world commercial environments. Summary of the Invention

[0005] The present invention aims to provide a scalable and accurate privacy-preserving large language model inference method and system to solve the problem that existing privacy-preserving inference technologies have failed to find a feasible balance between privacy, usability and scalability, and most solutions require modification of the LLM architecture or leakage of model parameters, making them difficult to deploy in practical commercial black-box API service scenarios.

[0006] To solve the above problems, the present invention adopts the following technical solution: Option 1: A scalable and accurate privacy-preserving inference method for large language models, performed by a noise sampler, comprising the following steps: Receive queries from one or more users, the queries containing general instructions and encrypted or labeled private attributes; Queries with similar semantics are grouped together to form batches based on text embedding technology; For each set of queries, obtain a set of candidate alternative attributes for the private attributes corresponding to each query, sample alternative attributes from the set of candidate alternative attributes to generate a noisy query, and mix the noisy query with the corresponding original query to form a batch request; The batch request is sent to the service provider's black-box large language model, and the batch response returned by the black-box large language model is received. The responses corresponding to noisy queries are identified and discarded from the batch responses, and the responses corresponding to the original queries are forwarded to the corresponding users.

[0007] Beneficial effects: The noise sampler mixes the original query with noisy queries to form batch requests, making it impossible for service providers to distinguish the authenticity of a single query from the batch requests, thus effectively protecting user input privacy. At the same time, since the original query itself is not modified but is directly fed into the black-box large language model to perform standard inference, the output quality is fully guaranteed, fundamentally solving the trade-off between privacy and utility in traditional privacy protection methods.

[0008] This invention constructs a privacy-preserving inference framework that maintains output quality without modifying the model architecture and is scalable. This invention provides a practical, scalable, plug-and-play privacy protection solution for scenarios using commercial black-box LLMAPIs, without relying on service provider collaboration or sacrificing output quality.

[0009] Furthermore, the grouping of semantically similar queries includes: Extract the embedding vectors of the common parts of each query, and calculate the similarity between the embedding vectors; Queries with similarity exceeding a preset threshold are grouped into the same group, or the most similar queries are grouped into the same group.

[0010] Beneficial effects: By grouping queries based on semantic similarity, queries in the same group have similar general instruction structures, which allows multiple users to share the same set of noisy query generation costs. This changes the privacy protection overhead from linear growth for a single user to amortization among multiple users. As the user base expands, the marginal privacy cost approaches zero, significantly improving the scalability of the solution.

[0011] Furthermore, the method for generating the candidate alternative attribute set includes: Receive encrypted private attributes and their corresponding candidate alternative attribute set sent by the user, wherein the candidate alternative attribute set is sampled by the user from a pre-built attribute database; or The system receives a mask tag sent by the user and uses a local large language model combined with a general instruction context to generate the set of candidate alternative attributes.

[0012] Beneficial effects: It provides flexible and diverse noise generation methods, supporting both deterministic sampling based on structured knowledge bases to ensure semantic controllability and generative sampling that leverages the capabilities of large language models to enhance adaptability to open domain attributes, enabling noise samplers to adapt to the privacy protection needs of different application scenarios.

[0013] Furthermore, when sampling alternative attributes from the candidate alternative attribute set, a (c,l) diversity constraint is introduced: For a candidate set S containing real and alternative attributes, let Pr(a|T) be the inferred probability of candidate attribute a∈S given a general instruction T. Sort the probability distributions such that... Then, forced satisfaction

[0014] Where c is the diversity parameter and l is the constraint length.

[0015] The constraint ensures that the probability of the top l-ranked high-probability candidate attribute does not exceed c times the probability of the lowest-probability candidate attribute ranked |S|.

[0016] Beneficial effects: The diversity constraint of this invention (c,l) requires the candidate attribute with the highest probability. Cannot compare rankings The total probability of the tail candidates is much higher than that of the head candidates. The technical essence is to limit the excessive concentration of high-probability candidate attributes in the top-ranked candidates, ensure that the probability quality of the head does not significantly exceed that of the tail candidates, and make the candidate attributes evenly distributed in probability distribution, thereby increasing the difficulty for attackers to distinguish the real attributes based on probability inference.

[0017] By using mathematical constraints to ensure a more uniform distribution of attributes within the candidate set, attackers are prevented from inferring the true attributes by exploiting the imbalance of probability distribution. This strengthens the rigor of privacy protection from an information theory perspective, making noisy queries appear as uniformly distributed rather than clustered in the semantic space, thus increasing the difficulty for attackers to distinguish them.

[0018] Furthermore, the sampling of alternative attributes to generate a noisy query also includes: Based on an exponential mechanism, a random number of copies are injected into each attribute combination. Specifically, for each attribute combination... Add a random number Extract from the following one-sided exponential distribution ,

[0019] Among them, probability density function and privacy parameter Related, followed by the attribute combination. injection One example.

[0020] Beneficial effects: By introducing a random number of replicas through an exponential mechanism, the frequency distribution of attribute combinations observed by the service provider satisfies the differential privacy guarantee of the neighbor relaxation, preventing attackers from inferring the real query through frequency pattern analysis. Even if a real attribute appears multiple times in the same batch, its frequency characteristics are overwhelmed by the random replicas of the noisy attributes, thus achieving privacy protection at the frequency level.

[0021] Furthermore, the step of mixing noisy queries with corresponding original queries to form batch requests also includes: A dynamic strategy for determining batch composition includes setting a waiting time window, where queries collected within the waiting time belong to the same batch, and queries exceeding the waiting time are assigned to the next batch. If the number of queries collected within the waiting time is lower than a preset threshold, the waiting time will be extended until the number of collected queries reaches the minimum threshold or the waiting time exceeds the upper limit.

[0022] Beneficial effects: By using a dual control mechanism of time window and threshold, a dynamic trade-off is achieved between batch latency and privacy costs. This avoids insufficient noise cost amortization due to too few queries, and prevents users from waiting too long by controlling the upper limit. It ensures scalability while taking into account user experience.

[0023] Furthermore, before receiving queries from one or more users, the process also includes: Receive private attributes encrypted with the service provider's public key, communication content encrypted with the noise sampler's public key, and private seeds encrypted with the service provider's public key sent by the user; When sending batch requests to the service provider, the private attributes in the noisy query are encrypted using the service provider's public key.

[0024] Beneficial effects: A secure communication protocol for the forward transmission phase was constructed, ensuring that the noise sampler completes noise query generation with only general instructions and cannot obtain any sensitive attribute information. This achieves a zero-knowledge state of user privacy for the noise sampler, so even if the noise sampler is compromised, attackers cannot obtain sensitive user data.

[0025] Furthermore, the step of identifying and discarding responses corresponding to noisy queries from batch responses and forwarding responses corresponding to original queries to the corresponding users also includes: Receive a masked batch response from the service provider, the mask being derived by the service provider from a pseudo-random generator based on a user-private seed; The original query response, after being masked, is directly forwarded to the corresponding user, who then reconstructs the original response using a modulo-subtraction operation based on their private seed.

[0026] Beneficial effects: During the reverse transmission phase, the noise sampler acts only as a relay node for the masked response and cannot read the response content, thus achieving a dual zero-knowledge state where the noise sampler has neither sensitive input nor sensitive output; at the same time, the seed-based masking mechanism avoids the direct exposure of user identity to the service provider, thereby achieving user anonymity protection.

[0027] Option 2: A scalable and accurate privacy-preserving large-scale language model inference method, executed on the user side, including the following steps: Identify and label the private attributes in the original query, and separate the private attributes from the general instructions; The general instructions are standardized and simplified to reduce reasoning costs; Encrypted or labeled private attributes and general instructions are sent to the noise sampler, which then performs the method described in any of the preceding items; Receive the response forwarded by the noise sampler. If the response has been masked, reconstruct the original response using a private seed or key, and verify the integrity of the response using a signature scheme.

[0028] Beneficial effects: On the user side, privacy-preserving calls to the black-box large language model service can be made through lightweight attribute annotation, query simplification and response reconstruction operations, without the need for local deployment of large models or complex cryptographic calculations, which significantly reduces the resource requirements of the client; at the same time, since the original query is not modified but directly used in inference, the output of the large language model obtained by the user has the same accuracy as in the unprotected scenario.

[0029] Option 3: A scalable and accurate privacy-preserving large-scale language model inference system, comprising: The query preprocessing module is deployed on the user side and configured to receive the original query input by the user, identify and label the private attributes in the original query, separate the private attributes from the general instructions, standardize and simplify the general instructions to reduce inference costs, encrypt the private attributes in the deployment mode where the noise sampler is a non-colluding third party, and send the processed query to the noise sampler. The grouping module, deployed on the noise sampler side, is configured to receive queries from one or more users, extract the embedding vectors of the common parts of each query, and group semantically similar queries into batches based on the similarity between the embedding vectors. The noise query sampling module is deployed on the noise sampler side. It is configured to obtain a set of candidate alternative attributes for each query in each group of queries, sample alternative attributes from the set of candidate alternative attributes to generate noise queries, introduce (c,l)-diversity constraints to ensure uniform distribution of candidate attributes, and inject a random number of copies for each attribute combination through an exponential mechanism. The noise queries are mixed with the corresponding original queries to form batch requests, and the batch requests are sent to the service provider. The response generation module, deployed on the service provider side, is configured to use a black-box large language model to perform standard inference on all queries in the received batch requests and return batch responses. In the deployment mode where the noise sampler is a non-colluding third party, it decrypts the encrypted private attributes in the query and generates a pseudo-random mask based on the private seed provided by the user to mask the response. The response forwarding module is deployed on the noise sampler side and is configured to receive batch responses returned by the service provider, identify and discard the responses corresponding to the noise query from the batch responses, and forward the responses corresponding to the original query to the corresponding users. The response receiving module, deployed on the user side, is configured to receive responses forwarded by the noise sampler. If the response is masked or encrypted, the original response is reconstructed using a private seed or key, and the integrity of the response is verified using a signature scheme.

[0030] Beneficial Effects: The six-module, three-party separation architecture achieves complete decoupling of privacy protection logic from the large language model. The query preprocessing module and response receiving module are deployed on the user side, performing lightweight attribute annotation, query simplification, and response reconstruction. The grouping module, noisy query sampling module, and response forwarding module are deployed on the noise sampler side, handling complex semantic grouping, noise generation, and response filtering. The response generation module is deployed on the service provider side, performing only standard black-box inference. All privacy protection components are encapsulated outside the LLM, requiring no modification to the model architecture and having no access to model parameters. For service providers, it is merely a regular batch API call, achieving true plug-and-play deployment. Simultaneously, because the original query is not modified but directly fed into the black-box large language model for standard inference, output quality is fully guaranteed. The batch processing mechanism distributes the cost of noisy queries among users in the same group, and as the user base expands, the marginal privacy cost approaches zero, providing the first practical and scalable privacy protection solution for commercial black-box LLM API services.

[0031] The advantages of this invention are: The core advantage of this invention lies in the fact that it achieves a unity of privacy, usability, and scalability for the first time, which is a triangular contradiction that cannot be achieved in the prior art.

[0032] Traditional solutions either offer strong privacy at the expense of efficiency, like secure multi-party computation, maintain efficiency but reduce utility, like cue word cleanup, or oscillate between the two, like differential privacy. This invention, through obfuscation and batch processing paradigms, losslessly mixes the original query into a set of noisy queries, making it impossible for the server to identify the true query from batch requests. Simultaneously, since the original query itself remains unmodified, the large language model performs a completely standard inference process, theoretically guaranteeing the output quality. This shift in privacy protection from modifying input to hiding input is the unexpected aspect of how this invention addresses its technical problem.

[0033] From an unexpected technical perspective, existing technologies generally follow the intuitive approach of protecting original data, namely, achieving privacy protection through encryption, perturbation, or replacement of original sensitive information. This invention counterintuitively adopts a strategy of protecting the location of the original data rather than its original form. The original query exists in plaintext, but its authenticity is submerged among a large number of semantically similar noisy queries. This philosophy of concealment echoes the steganography principle in cryptography, but represents an unprecedented technical approach in the field of privacy protection for large language models. More ingeniously, this invention utilizes a batch processing mechanism to socially distribute privacy overhead: the cost of a single user's noisy query is shared by multiple users in the same batch. As the user base grows, the marginal query cost approaches zero. This economic model design decouples high privacy protection from high costs, fundamentally solving the scalability bottleneck.

[0034] Furthermore, this invention achieves complete decoupling of the privacy protection module from the large language model in its system architecture. All privacy protection logic is encapsulated on the user side and in a separate noise sampler. For LLM service providers, what they receive is only a normal batch API call, without requiring any modification to the model architecture, parameters, or processes. This black-box compatibility means that this invention can seamlessly integrate with existing commercial LLM API services without requiring deep cooperation or authorization for architectural modifications from service providers. This has a decisive deployment advantage in real-world business environments and is another unexpected aspect of the technical means of this invention. Attached Figure Description

[0035] Figure 1 The diagram shows the overall system framework of the present invention.

[0036] Figure 2 This is a schematic diagram illustrating the implementation framework of the system of the present invention when multiple organizations are involved.

[0037] Figure 3 This diagram illustrates secure communication between a user and a service provider when the noise sampler is a non-colluding third party. Detailed Implementation

[0038] The following detailed description illustrates the specific implementation method: This invention provides a scalable and accurate privacy-preserving large-scale language model inference system that protects user input data during the LLM inference phase without requiring any modifications to the LLM architecture. Specifically, it allows users to keep their data privacy locally while using LLM services. On the other hand, service providers are often reluctant to modify the LLM structure or disclose model parameters for commercial reasons. To address these challenges, this design employs an obfuscation strategy to mask sensitive information in the original prompts and distributes inference costs by batch processing user requests.

[0039] A scalable and accurate privacy-preserving large-scale language model inference system, whose framework includes three participating parties: (1) User: Holds private data that needs to be kept confidential from service providers; (2) Noise Sampler: Responsible for sampling noise queries to obfuscate the original prompts. The noise sampler can be a third-party service provider offering privacy-preserving computing or an organization handling confidential queries from internal customers or employees; (3) Service provider: Deploy LLM and generate a response for each query.

[0040] The system comprises six modules: two deployed on the user side, three on the noise sampler side, and one on the service provider side. These modules are described below in operational order: (1) Query preprocessing module: Deployed on the user side. The user identifies and labels the private attributes in the prompts, and standardizes and simplifies the prompts to reduce reasoning costs. Simplification can be achieved through prompt compression (condensing and summarizing the prompt content) or decomposition (breaking down complex queries into simple sub-tasks). If the noise sampler is a non-colluding third party, the user can also encrypt the private attributes in the prompts to protect sensitive information against third parties.

[0041] (2) Grouping Module: Deployed on the noise sampler side. The noise sampler groups semantically similar cues to achieve noise cue sharing. The grouping operation can be completed through the embedding vectors extracted by the sentence converter.

[0042] (3) Noise Query Sampling Module: Deployed on the noise sampler side. This module samples alternative attributes to obfuscate the original private attributes and generates pseudo queries by replacing the original attributes. Ultimately, the server can only observe the mixture of the original query and the noise (or pseudo) query.

[0043] (4) Response Generation Module: Deployed on the service provider side. This module uses LLM to generate a response for each query (including the original query and the noisy query). If the noise sampler is a non-colluding third party, the service provider needs to decrypt the encrypted privacy information in the query and encrypt or mask the response to prevent user information from being leaked to the noise sampler.

[0044] (5) Response forwarding module: Deployed on the noise sampler side. After receiving the message from the service provider, the noise sampler discards the response corresponding to the noise query and forwards the original query response (which may be masked or encrypted) to the corresponding user.

[0045] (6) Response receiving module: Deployed on the user side. If the response is masked or encrypted, the user can reconstruct the response using a private seed or key. The user can also verify the integrity of the response using a signature scheme.

[0046] The existing technologies involved in the module are described below: Text embedding: Transforming text into a numerical vector representation using a converter model. This vector representation can be used for downstream tasks such as semantic similarity calculation, cluster analysis, and recommender systems.

[0047] Asymmetric encryption: uses a public key and a private key pair to encrypt and decrypt data. The public key can be publicly shared for encrypting messages, while the private key is kept strictly confidential for decrypting messages.

[0048] Digital signatures: used as a cryptographic method to verify the authenticity and integrity of messages. In this framework, digital signatures ensure that the response was indeed generated by the service provider and has not been tampered with.

[0049] Based on the above system, the scalable and accurate privacy-preserving large-scale language model inference method of the present invention includes the following steps: On the user side, private attributes in the original query are marked, and the original query containing private attributes is separated from the general command. At the same time, the query is standardized and simplified to reduce the reasoning cost. The noise sampler uses text embedding technology to group semantically similar queries. For the private attributes in each group of queries, it samples alternative attributes from the candidate attribute set to generate noise queries, and mixes the original queries with the noise queries to form batch requests. The service provider uses a black-box large language model to perform inference on all queries in a batch of requests and returns a response; the noise sampler discards the responses to noisy queries and forwards the responses to the original queries to the corresponding users. The generation of the noise query introduces a (c,l)-diversity constraint to ensure that the candidate attributes are evenly distributed, and injects a random number of copies of each attribute combination through an exponential mechanism so that the service provider's observations satisfy the differential privacy distribution.

[0050] like Figure 3 As shown in the diagram, this method illustrates secure communication between the user and the service provider when the noise sampler is a non-colluding third party. First, the user encrypts private attributes and a private seed using the server's public key to ensure the noise sampler cannot access the privacy information. Then, the server decrypts the message using its own private key to obtain the plaintext prompt and the private seed. After generating the response, the service provider generates a random vector using the user's seed and adds it to the response to achieve response masking. Finally, the user reconstructs the original response using the private seed.

[0051] This method achieves privacy protection by transmitting the original query to the server without loss and mixing in noisy queries, making it impossible for the server to distinguish the authenticity of a single query. At the same time, since the original query is not modified, the output quality of the large language model is fully guaranteed, fundamentally solving the trade-off between privacy and utility in traditional methods.

[0052] Some technical details of this method can be limited as follows: 1. Noise query generation mechanism: This can be executed on the user's end (assuming the noise sampler is untrusted by the user), or executed by the noise sampler (assuming the noise sampler is trusted by the user). The generated content includes, but is not limited to: Maintain a database of private attributes. Each attribute category contains a series of private attributes, and users sample attributes from that category, such as: {"job": ["cybersecurity engineer", "attorney", ...]} Replace the private attribute with #MASK, and let the large model combine the context to give a series of attributes that #MASK may correspond to.

[0053] After encrypting the private attributes, the user sends the candidate set (including the replacement attributes of the original private attributes) corresponding to each encrypted attribute to the noise sampler. The noise sampler samples the set of multiple private attributes and fills the sampled set into the public instruction.

[0054] To control the similarity between noisy queries and true queries, a (c,l) diversity constraint can be introduced. For a candidate set S containing true attributes and alternative attributes, let Pr(a|T) be the candidate attribute given a general instruction T. The inferred probability. Sort the probability distributions so that... Then, forceful fulfillment:

[0055] This ensures that the attributes are distributed more evenly within the candidate set.

[0056] 2. Quantitative design of confusion ratio: Assuming each set of real prompt words contains Each private attribute Corresponding to A set of replacement attributes. If all sets of replacement attributes are sampled, it will produce... Complexity. To reduce query complexity, we will sample the replacement attribute set, thus reducing the final sample size. satisfy:

[0057]

[0058] Based on the above sample size, for each private attribute, there are no fewer than The probability that all replacement attributes appear in the final sample combination.

[0059] To determine each attribute The corresponding number of replacement attributes The number of replacement attributes can be determined based on their frequency in the keyword database or in the actual context. For attributes that are more general and comprehensive, a lower number of replacement attributes (e.g., 10) can be used; for attributes that are more unique and specific, a higher number of replacement attributes (e.g., 40) can be used.

[0060] 3. Details of the SMPC protocol for batch processing cost sharing (assuming the noise extractor is untrusted by the user): The protocol aims to ensure that: (i) the noise sampler does not possess any sensitive data and only observes general instructions; and (ii) the service provider's view consists of an anonymous, scrambled set of cue words, including both real and noisy cue words. A core design challenge is to ensure secure communication between the user, the noise sampler, and the service provider while maintaining the user's anonymity with the service provider.

[0061] (1) Forward transmission The forward phase transmits messages from the user to the service provider. During this phase, the user uses the service provider's public key. Private attributes are encrypted, and the service provider uses its private key. The ciphertext is decrypted. This asymmetric encryption scheme hides private properties from any third party (including the noise sampler). After sampling and generating a noise query, the noise sampler also decrypts the private properties in the noise query. Encryption is performed. Furthermore, any messages sent to the noise sampler use that sampler's public key. Encryption is used to prevent eavesdroppers (including service providers) from obtaining the content. The above encryption methods can be implemented using RSA for asymmetric encryption.

[0062] (2) Reverse transmission The reverse phase transmits the response from the service provider to the user. This response may contain information related to private attributes, therefore the noise sampler must be unaware of its content. This phase requires the user to be able to decrypt the response without exposing their identity ID to the service provider. To achieve this, we devise a mask-based obfuscation scheme: (i) the user sends a seed encrypted using the service provider's public key. (ii) The service provider decrypts the seed. And derive a pseudo-random mask using a pseudo-random generator (AES can be used). The mask is obtained through modulo addition. To obfuscate the response. (iii) Upon receiving Then, the user performs a modulo subtraction operation. The original response is reconstructed. This represents the sequence of token IDs from the original answer. It is a pseudo-random mask sequence. This represents the sequence of answer token IDs following the mask.

[0063] The above multi-party security computing mechanism requires that noise samplers and service providers not collude to steal user information.

[0064] 4. The mathematical community regarding the probability of privacy leaks The following is a theoretical analysis of the optimal attack success rate (ASR).

[0065] make express The original query from the candidates, and set This represents everything the attacker observes (e.g., candidate sets, general instructions, and any available context). Based on Bayesian rules, the attacker's optimal choice is:

[0066] The second equation is derived from Bayes' theorem: for all Their normalized denominators are the same. Let (Attack success rate) represents the attacker's optimal attack success rate, i.e., the optimal probability of successfully predicting the original query. Under this optimal rule, the best achievable success probability is:

[0067] in Indicates to The expected average of the distribution is calculated. Then, its upper bound can be derived as:

[0068] in Indicates to Calculate the expected average of the distribution. This indicates the number of queries in the candidate set. This limit highlights two key factors in ASR: Number of candidates: Under the same conditions, when the number of newly added candidates is in a given... It is still reasonable under the circumstances (i.e.) A larger candidate set will reduce the achievable accuracy.

[0069] Variance of the posterior distribution: If all candidates remain similarly reasonable, then It will grow larger, thus making the upper bound tighter.

[0070] 5. Provide a simplified algorithm. Compression technologies (including but not limited to): 1) Use a lightweight model to extract the user's core needs, and strip away redundant context and ineffective conversational pleasantries; 2) By leveraging the underlying attention weighting mechanism, keywords with high information entropy are accurately retained.

[0071] Decomposition techniques (including but not limited to): Train a local decomposition model to deconstruct complex, long prompts into a sequence of atomic tasks with logical order.

[0072] 6. Dynamic strategy for batch composition Batch size is determined by the number of users collected within a fixed time period. A waiting time is initially set; all queries collected within this waiting time belong to that batch. Queries exceeding the waiting time are moved to the next group. Collected queries are grouped based on their public portion (the encrypted portion containing private information), with queries having similar public portions grouped together.

[0073] To prevent attackers from inferring privacy information through side-channel data such as response length and time, the service provider pads all outputs within the same group to the same length and applies a mask value. The masked responses from each group are then transmitted to the noise sampler at a constant rate.

[0074] 7. Black-box calling method of cloud-based LLM (1) Service provider changes (incompatible with standard API) The service provider needs to add the following functions: (a) decrypt encrypted content; (b) identify identical queries and deduplicate them (inferring only once) to reduce query costs; and (c) mask the response.

[0075] Since the existing API does not have the above functionality, the service provider needs to modify the existing API.

[0076] (2) The service provider does not make any changes (compatible with the standard API) A third party needs to be added between the noise sampler and the service provider. This third party must not collude with the noise sampler to steal user information. This third party will perform the following functions in sequence: (a) decrypt encrypted content (the user encrypts the content using the third party's public key during forward transmission); (b) identify duplicate queries and deduplicate them; (c) directly call the service provider's API to perform a query; (d) obtain the service provider's response and mask it; (e) match the masked response with the query and return it to the noise sampler.

[0077] 8. Architectural differences between user-level and organization-level deployments like Figure 1 As shown, the system of this invention is deployed at the user level. This architecture includes six modules, which are deployed among the user, the noise sampler, and the service provider, respectively. The third-party proxy node corresponds to the noise sampler.

[0078] like Figure 2 As shown in the diagram, this is a deployment diagram at the organizational level, where each organization hosts several internal users. The query preprocessing module, grouping module, noise query sampling module, and response forwarding module are all deployed on the organization's server.

[0079] Organizational-level deployments are carried out in batches through third-party collaboration among various organizations.

[0080] Differences in trust models: User-level deployment strictly requires that the noise sampler and the service provider not collude; if they do, all user information will be leaked.

[0081] Even if third-party service providers collude in an organization-level deployment, the noise sampling conducted within each organization can still protect each user's information to some extent.

[0082] 9. Reuse and update mechanism for noisy queries Noise queries are used only once. Each time, the user generates (according to the method shown in "1. Noise Query Generation Mechanism") a replacement attribute corresponding to each private attribute, and the noise sampler performs combined sampling.

[0083] 10. Equivalence Analysis of Obfuscation Mechanisms and Differential Privacy After sampling the noise combinations, the noise sampler needs to sample based on frequency. This is to prevent the server from inferring the original information through frequency distribution. If the sampled noise combinations are directly added to the query set proportionally, a situation might arise where a real attribute appears multiple times in the same batch, while a forged attribute appears only once. An attacker could then distinguish the real query based on frequency patterns. Furthermore, because privacy attributes are encrypted, the noise sampler cannot guarantee that the frequency of the noise attributes will match the frequency of the real attributes.

[0084] To alleviate this problem, a sampling method based on an exponential mechanism is proposed. Instead of injecting each qualified combination exactly once, it injects each attribute combination... Add a random number Specifically, for each The noise sampler draws from the following one-sided exponential distribution. :

[0085] in For privacy parameters. Subsequently, the sampler will perform a sample for each combination. injection One example.

[0086] The above sampling ensures that the service provider's observation horizon satisfies a relaxed differential privacy distribution (neighborhood). We define a... -Neighbor: Will and These are respectively recorded as user prompts Privacy attributes and general instructions (general instructions refer to the part of the prompt that deletes privacy attributes). [The following appears to be a separate, unrelated sentence:] Command for The set of qualified candidate attributes. For two sets of hints... and ,if By A certain prompt Privacy attributes If obtained through deletion, it is called... and for -Neighbor. We remember it as .

[0087] Then for any ,satisfy -DP:

[0088] 11. Suggest simplified semantic preservation evaluation metrics Using existing technology, calculate the cosine of the angle between the vectors of the original query and the simplified query after passing through a text encoder (such as Sentence-BERT).

[0089] 12. Edge Case Handling Mechanism Using existing technology, a waiting time is first determined. All queries collected within this waiting time (e.g., 10 seconds) belong to this batch. If the waiting time is exceeded, the queries are assigned to the next group. If the number of queries collected within this waiting time is lower than a certain threshold, the waiting time will be extended until one of the following conditions is met: (1) the number of queries collected reaches the minimum threshold; (2) the waiting time exceeds the upper limit (e.g., 1 minute).

[0090] The differences and advantages of this invention compared with prior art document 1 (CN118410522A: Privacy Data Protection Method, Apparatus, Medium and Device for Large Language Models): Comparative document 1 discloses a privacy-preserving data method based on adversarial feature extraction networks. Its core lies in generating privacy-preserving vectors through adversarial training within a large language model. Local privacy data is then converted into irreversible feature vectors and stored in a client-side database. During inference, question prompts are constructed by matching the question feature vectors with the privacy-preserving vectors. Essentially, this approach involves feature substitution, replacing the original privacy text with an irreversible vector representation.

[0091] This invention differs fundamentally from prior art document 1. First, the technical approaches are completely different: prior art document 1 relies on the adversarial feature extraction network within the LLM, requiring access to the model's internal structure for feature extraction and adversarial training, effectively demanding white-box or gray-box access. In contrast, this invention places all privacy-preserving logic outside the LLM, treating it as a pure black box, performing batch queries only through a standard API interface, without any access to the model's internal structure. Second, the privacy protection mechanisms differ: prior art document 1 achieves protection by converting privacy data into irreversible vectors, but vector representation may lose semantic details, and the security of the adversarial feature extraction network depends on the sufficiency of adversarial training. This invention, however, does not modify any content of the original query, but hides the location of the real query through the crowdsourcing effect of noisy queries, preserving 100% of the semantic integrity of the original query. Third, the deployment modes are vastly different: prior art document 1 requires deploying a local privacy knowledge database and adversarial feature extraction network on the client side, suitable for local knowledge augmentation scenarios such as RAG; this invention is geared towards online API service scenarios, where users do not need to store the privacy knowledge base locally or run large models locally for feature extraction, significantly reducing client resource requirements. Finally, the batch processing mechanism of the present invention has a cost-sharing advantage that is completely absent in the comparison document 1. In the comparison document 1, each user performs feature extraction and vector matching independently, and cannot reduce privacy overhead through economies of scale.

[0092] Differences and advantages of this invention compared to prior art document 2 (CN119128984A Privacy Protection Method, Apparatus, Device, Medium and Product in Large Model Inference): Comparative document 2 discloses a privacy-preserving method based on perturbation transformation and a local decoder. This method performs perturbation transformation on each original statement in the original query command, selects the optimal perturbation statements through semantic coherence evaluation to form a perturbation dataset, sends it to a cloud-based LLM (Local Level Management Model), and then inputs the returned perturbation query results into a pre-trained local decoder to restore the target query result. The essence of this scheme is perturbation and restoration; first, controllable noise is added to the input, and then the output noise is cleaned by the decoder.

[0093] The core difference between this invention and prior art document 2 lies in their approach to processing the original query. Prior art document 2 still modifies the original query, replacing it with a perturbated statement. Although semantic coherence filtering is used to maintain semantic similarity as much as possible, a semantic deviation inevitably exists between the perturbated query and the original query. This deviation directly affects the accuracy of LLM inference, and the local decoder's restoration capability has a theoretical upper limit, making it impossible to guarantee 100% restoration of the original semantics. This invention, however, adheres to the principle of zero modification to the original query. The original query enters the LLM inference process in its completely authentic form, and the LLM performs standard, unbiased inference on it. The output quality is theoretically completely consistent with scenarios without privacy protection. This is the decisive advantage of this invention in terms of usability.

[0094] Furthermore, the privacy protection strength of Comparative Document 2 depends on the contradiction between the noise intensity of the perturbation transform and the reconstruction capability of the local decoder: stronger noise results in better privacy but makes reconstruction more difficult, while weaker noise makes reconstruction easier but reduces privacy. This invention does not exhibit this inherent contradiction: the privacy protection strength is determined by the number and diversity of noisy queries, while the output quality is completely independent of the noisy queries, achieving decoupling optimization. At the architectural level, Comparative Document 2 requires users to deploy and train the decoder model locally, increasing the client's computational burden; the user side of this invention only needs to perform lightweight operations such as annotation, encryption, and response reconstruction, while the complex noise sampling and batch processing scheduling are handled by the noise sampler, achieving client-side lightweighting. Finally, Comparative Document 2 does not involve multi-user batch processing and cost-sharing mechanisms, and its privacy protection overhead increases linearly with the query; the batch processing strategy of this invention achieves sublinear or even logarithmic overhead growth by sharing the noise query cost among multiple users, providing an order-of-magnitude advantage in scalability.

[0095] We compared the accuracy (F1) of the method of this invention with that of the method in reference document 2 on the MMLU (Massive Multitask Language Understanding, http: / / github.com / hendrycks / test) dataset, using GPT-4o-mini and GPT-4o as cloud query models, and the local decryptor in reference document 2 using a finely tuned Qwen3-4B. As shown in Table 1, the method of this invention significantly improves accuracy compared to the reference document.

[0096] Table 1

[0097] Furthermore, in professional scenarios such as medical diagnosis, legal review, and financial risk control, the private attributes in user queries are often not dispensable supplementary information, but rather key factual conditions that determine the conclusions of reasoning. For example, age, symptom duration, and past medical history in medical scenarios; contract parties, transaction targets, and statements of rights and obligations in legal scenarios; and holding ratios, asset classes, and market indicators in financial scenarios, all directly affect the judgment path and output conclusions of the large language model. If the perturbation transformation method described in Comparative Document 2 is used, even if the perturbed text remains semantically similar, subtle shifts in key facts may lead to substantial deviations in diagnostic recommendations, legal opinions, or risk assessment results. Therefore, in the aforementioned high-precision, high-responsibility scenarios, industry applications have a rigid requirement for zero semantic deviation; it is not feasible to simply perturb the original query into a similar query and then rely on the local decoder for recovery.

[0098] The differences and advantages of this invention compared to prior art document 3 (CN117725610A A privacy-preserving proxy method for third-party large language models): Prior art document 3 discloses a privacy-preserving proxy method for third-party large language models. Its core is to privatize the original prompt words by customizing a local large model, replacing privacy keywords with identifiers or encrypted values ​​to generate secure prompt words for interaction with the third-party LLM. After the interaction, deprivation and restoration are performed. Essentially, this scheme involves keyword replacement, using placeholders to replace sensitive entities.

[0099] The difference between this invention and prior art document 3 lies primarily in the granularity and mechanism of privacy protection. Prior art document 3's privacy processing targets predefined privacy keyword types (such as names, phone numbers, ID cards, etc.) and employs fixed identifier replacement rules. This one-size-fits-all replacement method alters the true semantics of the prompts; for example, replacing disease names in a medical context might lead to diagnostic suggestions deviating from the actual condition. This invention, however, targets private attributes at any semantic level, obfuscating them by generating semantically similar noisy queries without modifying any words in the original query, thus preserving the complete semantics of the input. Secondly, prior art document 3 relies on a locally deployed, customized large model for privacy processing and de-privacy recovery. This requires users to have the computational resources to run a large model locally, and the processing power of the local model directly determines the quality of privacy protection. This invention delegates complex noise sampling and group scheduling to a noise sampler, requiring only lightweight annotation and encryption operations on the user side, resulting in extremely low client resource requirements.

[0100] More importantly, the privacy protection in Comparison Document 3 is a deterministic substitution; once an attacker learns the substitution rules or breaks the local model, they can completely restore the privacy information. The obfuscation mechanism of this invention, however, is a probabilistic hiding mechanism. The real query is submerged in a large number of semantically plausible noisy queries. Even if an attacker fully understands the noise generation algorithm, they cannot deterministically identify the real query from a probabilistic perspective. Its privacy protection possesses robustness at the information theory level. Furthermore, Comparison Document 3 does not involve any form of batch processing or cost optimization mechanism; each query undergoes independent privacy processing and API calls. The batch processing mechanism of this invention achieves query aggregation through time windows and semantic grouping, distributing the cost of noisy queries across all users in the same group, thus achieving sustainable privacy protection from an economic perspective.

[0101] The differences and advantages of this invention compared to conventional techniques in the field are as follows: Conventional techniques in privacy-preserving inference include homomorphic encryption, secure multi-party computation, differential privacy, and federated learning. Compared to homomorphic encryption and secure multi-party computation, this invention does not require modification of the model's forward propagation computation process, does not require the model holder to participate in key management or collaborative computation, has full compatibility with black-box LLM APIs, and avoids the orders-of-magnitude computational overhead and communication latency introduced by cryptographic schemes. Compared to differential privacy, this invention does not directly add calibration noise to the input, but hides the real input through an external obfuscation strategy, avoiding the direct impact of noise on the model output and achieving complete decoupling between privacy protection strength and output quality. Compared to federated learning, this invention does not involve distributed model training, focusing only on privacy protection during the online inference phase, making its application scenarios more focused and significantly reducing deployment complexity.

[0102] The obfuscation and batch processing paradigm of this invention represents an innovative technical approach in the field of privacy computing. Traditional privacy computing follows a framework of either data remaining stationary while the model moves, or vice versa. This invention proposes a new paradigm where the query remains stationary while the location moves. The data (query) itself is not encrypted, disturbed, or replaced, but its identity is hidden through batch obfuscation. This paradigm shift allows the privacy protection module to be deployed as an independent infrastructure layer at the front end of the LLM service, similar to a load balancer or API gateway, without needing to penetrate into the model itself. This offers fundamental advantages in engineering implementation and commercial deployment.

[0103] The specific implementation process is as follows: Example 1: Patient privacy protection scenario in medical institutions A top-tier hospital has deployed the system of this invention to protect patient privacy when using LLM-assisted diagnosis. A patient inputs the query: "A 45-year-old male patient, a cybersecurity engineer, has recently experienced persistent chest pain accompanied by shortness of breath. He has a history of hypertension. Please analyze possible causes and suggest examination plans." The query preprocessing module identifies and labels private attributes: age "45 years old", gender "male", occupation "cybersecurity engineer", symptoms "persistent chest pain with shortness of breath", and medical history "hypertension". The system separates general instructions ("Please analyze possible causes and suggest examination plans") from private attributes and compresses the query, removing redundant polite language and formatting marks.

[0104] The grouping module extracts the embedding vector through a sentence converter and groups the query with other cardiovascular disease-related queries within the same time period. The noise query sampling module samples alternative attributes from the attribute database: age is replaced with "38 years old", "52 years old", and "61 years old"; occupation is replaced with "lawyer", "accountant", and "teacher"; symptoms are replaced with "intermittent chest tightness", "palpitations after exercise", and "paroxysmal nocturnal dyspnea"; and medical history is replaced with "diabetes", "hyperlipidemia", and "post-coronary artery disease surgery". A (c,l)-diversity constraint ensures that each attribute is evenly distributed in the candidate set, ultimately generating a batch request containing one real query and 39 noise queries. An exponential mechanism injects a random number of copies into each attribute combination.

[0105] The service provider's black-box LLM performs standard inference on 40 queries, returning 40 sets of diagnostic suggestions. A noise sampler discards the responses to 39 noisy queries and returns the responses to the original queries (including an analysis of the etiology of chest pain for a 45-year-old male cybersecurity engineer) to the patient via masked transmission. The patient reconstructs the response using a private seed and verifies the signature integrity.

[0106] Effects and Advantages: In this embodiment, the service provider's view is an anonymous set of 40 semantically reasonable medical queries, making it impossible to determine which corresponds to a real patient. Since the original queries are not modified, the LLM performs completely standard medical reasoning on the real queries, and the accuracy of the diagnostic recommendations is completely consistent with scenarios without privacy protection. In contrast, traditional solutions using prompt word cleansing may cause the LLM to overlook occupational environmental exposure factors (such as the cardiovascular effects of prolonged sitting and high-stress work) if occupational information such as "cybersecurity engineer" is removed, leading to incomplete diagnostic recommendations. If the perturbation scheme in Comparative Document 2 is used, perturbing "persistent chest pain" to "intermittent chest pain" may guide the LLM to reason towards unstable angina rather than acute myocardial infarction, resulting in a risk of misdiagnosis. This invention completely avoids such usability losses through a "zero-modification" strategy. The batch processing mechanism allows the noise cost of the 40 queries to be shared among users in the same group, making the marginal query cost for a single user approach zero. Hospitals do not need to pay 40 times the API fee for each query, achieving scalable deployment.

[0107] Example 2: Financial Company Portfolio Risk Inquiry Scenario An asset management company uses the system of this invention to protect sensitive client attributes. An investment manager enters the query: "Client A holds 35% of its portfolio in the technology sector, with top holdings including a chip design company (12%) and a cloud computing company (8%). The current market volatility index (VIX) is 28. Please assess the portfolio's drawdown risk under extreme market conditions." The query preprocessing module marks private attributes: customer identifier "A", sector percentage "35%", names and percentages of heavily held stocks "chip design company 12%", "cloud computing company 8%", and VIX value "28". The system processes the numerical privacy attributes using a preset encryption algorithm and generates a candidate replacement set for the textual attributes.

[0108] The grouping module groups the query with other portfolio risk queries within the same time period based on semantic similarity. The noisy query sampling module generates replacement attributes: customer identifiers are replaced with "B", "C", and "D"; sector percentages are replaced with "28%", "42%", and "51%"; top holdings are replaced with other companies in the same industry and their random percentages; and VIX values ​​are replaced with "22", "35", and "41". Through obfuscation ratio quantification, it is ensured that each attribute has at least a 95% probability of appearing in the final sampling.

[0109] In an organizational-level deployment model, the company's internal servers act as noise samplers, while third parties are only responsible for batch coordination. Even if the third party colludes with the service provider, because noise sampling is completed internally, attackers can only obtain information about the overall query traffic related to technology sector investments within the company, and cannot pinpoint the specific holding details of customer A.

[0110] Effects and Advantages: This example demonstrates the advantages of an organization-level deployed trust model. If the proxy solution in Comparative Document 3 were deployed in this scenario, "chip design company" would need to be replaced with a fixed identifier such as "...".<mask_stock1> The risk assessment returned by LLM is based on abstract identifiers rather than the actual fundamentals of the companies, leading to distorted estimates of drawdown risk. Furthermore, the locally customized large model in Comparison Document 3 requires pre-training to identify financial entities, limiting its ability to identify emerging companies or non-standard holdings. This invention uses semantically similar noisy company names for obfuscation; LLM's analysis of real companies is entirely based on their actual business and market position, ensuring the accuracy of risk assessment remains unaffected. Organizational-level deployment, through a separate architecture of internal noise sampling and external batch collaboration, achieves a defense-in-depth effect where privacy is protected even if some trusted parties are compromised, meeting the stringent compliance requirements of the financial industry for data security.

[0111] Example 3: Contract Review Scenario in a Law Firm A law firm is using the system of this invention to process confidential client data. A lawyer enters the query: "Please review the following contract terms: Party A is a multinational pharmaceutical company, Party B is a CRO company, the contract involves the sharing of clinical trial data for a new drug, please identify the risk points in the intellectual property ownership clauses and suggest modifications." The query preprocessing module marks private attributes: the name of Party A is "a multinational pharmaceutical company", the name of Party B is "a CRO company", and the subject matter of the contract is "new drug clinical trial data sharing". The system decomposes long text queries, breaking down the contract review task into three atomic sub-tasks: "intellectual property ownership identification", "data sharing scope definition", and "breach of contract liability analysis", generating noisy query batches for each sub-task.

[0112] The noise query sampling module targets the company name attribute and samples alternative attributes from the pharmaceutical industry company database: Party A is replaced with the name of another real multinational pharmaceutical company, Party B is replaced with the name of another CRO company, and the subject matter is replaced with semantically similar but different contract types such as "biomarker sharing," "production process technology transfer," and "overseas clinical trial application agency." (c,l)-diversity constraints ensure that the candidate company names are evenly distributed in the semantic space, preventing attackers from inferring the true identity based on company reputation or industry status.

[0113] After the service provider returns responses in batches, a noise sampler filters the original query responses, and lawyers reconstruct the complete review opinion using a private seed. The system verifies the responses have not been tampered with via digital signatures, ensuring the integrity and traceability of the legal advice.

[0114] Effects and Advantages: This embodiment demonstrates the adaptability of the invention to complex professional scenarios. Contract information handled by law firms is highly commercially sensitive; its leakage could lead to stock price fluctuations or competitive disadvantages. The vector replacement scheme in Reference 1 cannot handle such structured legal texts because the semantics of contract terms are highly dependent on specific entity names and precise legal expressions; vector representation would lose crucial party identities and subject matter details. The perturbation scheme in Reference 2 might perturb "intellectual property ownership" to "intellectual property licensing," causing LLM to shift from ownership analysis to licensing condition analysis, completely deviating from the lawyer's actual needs. This invention, by maintaining the complete authenticity of the original query, ensures that LLM performs professional legal analysis based on accurate party identities and contract subject matter, and the returned review opinions have professional value that can be directly used for client reporting. The decomposition technique breaks down complex contract review tasks into atomic subtasks, with each subtask's noisy query batch generated independently. This reduces the scale pressure of a single batch and increases the difficulty of reconstruction by attackers through subtask-level obfuscation, achieving a multiplicative effect on privacy protection strength.

[0115] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A scalable and accurate privacy-preserving inference method for large-scale language models, characterized in that, Performed by the noise sampler, the process includes the following steps: Receive queries from one or more users, the queries containing general instructions and encrypted or labeled private attributes; Queries with similar semantics are grouped together to form batches based on text embedding technology; For each set of queries, obtain a set of candidate alternative attributes for the private attributes corresponding to each query, sample alternative attributes from the set of candidate alternative attributes to generate a noisy query, and mix the noisy query with the corresponding original query to form a batch request; The batch request is sent to the service provider's black-box large language model, and the batch response returned by the black-box large language model is received. The responses corresponding to noisy queries are identified and discarded from the batch responses, and the responses corresponding to the original queries are forwarded to the corresponding users.

2. The method according to claim 1, characterized in that, The grouping of semantically similar queries includes: Extract the embedding vectors of the common parts of each query, and calculate the similarity between the embedding vectors; Queries with similarity exceeding a preset threshold are grouped into the same group, or the most similar queries are grouped into the same group.

3. The method according to claim 1, characterized in that, The methods for generating the candidate alternative attribute set include: Receive encrypted private attributes and their corresponding candidate alternative attribute set sent by the user, wherein the candidate alternative attribute set is sampled by the user from a pre-built attribute database; or The system receives a mask tag sent by the user and uses a local large language model combined with a general instruction context to generate the set of candidate alternative attributes.

4. The method according to claim 1, characterized in that, When sampling alternative attributes from the candidate alternative attribute set, a (c,l) diversity constraint is introduced: For a candidate set S containing real and alternative attributes, let Pr(a|T) be the inferred probability of candidate attribute a∈S given a general instruction T. Sort the probability distributions such that... Then, forced satisfaction Where c is the diversity parameter and l is the constraint length.

5. The method according to claim 1, characterized in that, The sampling substitution attribute to generate a noisy query also includes: Based on an exponential mechanism, a random number of copies are injected into each attribute combination. Specifically, for each attribute combination... Add a random number Extract from the following one-sided exponential distribution , Among them, probability density function and privacy parameter Related, followed by the attribute combination. injection An example.

6. The method according to claim 1, characterized in that, The step of mixing noisy queries with corresponding original queries to form batch requests also includes: A dynamic strategy for determining batch composition includes setting a waiting time window, where queries collected within the waiting time belong to the same batch, and queries exceeding the waiting time are assigned to the next batch. If the number of queries collected within the waiting time is lower than a preset threshold, the waiting time will be extended until the number of collected queries reaches the minimum threshold or the waiting time exceeds the upper limit.

7. The method according to claim 1, characterized in that, Before receiving queries from one or more users, the process also includes: Receive private attributes encrypted with the service provider's public key, communication content encrypted with the noise sampler's public key, and private seeds encrypted with the service provider's public key sent by the user; When sending batch requests to the service provider, the private attributes in the noisy query are encrypted using the service provider's public key.

8. The method according to claim 1, characterized in that, The step of identifying and discarding responses corresponding to noisy queries from batch responses and forwarding responses corresponding to original queries to the corresponding users also includes: Receive a masked batch response from the service provider, the mask being derived by the service provider from a pseudo-random generator based on a user-private seed; The original query response, after being masked, is directly forwarded to the corresponding user, who then reconstructs the original response using a modulo-subtraction operation based on their private seed.

9. A scalable and accurate privacy-preserving reasoning method for large-scale language models, characterized in that, Executed by the user, including the following steps: Identify and label the private attributes in the original query, and separate the private attributes from the general instructions; The general instructions are standardized and simplified to reduce reasoning costs; Encrypted or labeled private attributes and general instructions are sent to the noise sampler, which then performs the method as described in any one of claims 1 to 8; Receive the response forwarded by the noise sampler. If the response has been masked, reconstruct the original response using a private seed or key, and verify the integrity of the response using a signature scheme.

10. A scalable and accurate privacy-preserving large-scale language model inference system, characterized in that, include: The query preprocessing module is deployed on the user side and configured to receive the original query input by the user, identify and label the private attributes in the original query, separate the private attributes from the general instructions, standardize and simplify the general instructions to reduce inference costs, encrypt the private attributes in the deployment mode where the noise sampler is a non-colluding third party, and send the processed query to the noise sampler. The grouping module, deployed on the noise sampler side, is configured to receive queries from one or more users, extract the embedding vectors of the common parts of each query, and group semantically similar queries into batches based on the similarity between the embedding vectors. The noise query sampling module is deployed on the noise sampler side. It is configured to obtain a set of candidate alternative attributes for each query in each group of queries, sample alternative attributes from the set of candidate alternative attributes to generate noise queries, introduce (c,l)-diversity constraints to ensure uniform distribution of candidate attributes, and inject a random number of copies for each attribute combination through an exponential mechanism. The noise queries are mixed with the corresponding original queries to form batch requests, and the batch requests are sent to the service provider. The response generation module, deployed on the service provider side, is configured to use a black-box large language model to perform standard inference on all queries in the received batch requests and return batch responses. In the deployment mode where the noise sampler is a non-colluding third party, it decrypts the encrypted private attributes in the query and generates a pseudo-random mask based on the private seed provided by the user to mask the response. The response forwarding module is deployed on the noise sampler side and is configured to receive batch responses returned by the service provider, identify and discard the responses corresponding to the noise query from the batch responses, and forward the responses corresponding to the original query to the corresponding users. The response receiving module, deployed on the user side, is configured to receive responses forwarded by the noise sampler. If the response is masked or encrypted, the original response is reconstructed using a private seed or key, and the integrity of the response is verified using a signature scheme.

Images