Method and system for secure inference of large models with trusted execution environment and differential privacy

Abstract

The application discloses a large model security reasoning method and system fusing a trusted execution environment and differential privacy, and belongs to the technical field of artificial intelligence. The method comprises the following steps: dividing a large model into an embedding layer and a core processing layer, deploying the embedding layer in a local client SGX2 trusted environment, and deploying the core processing layer on a remote server equipped with a multi-GPU acceleration card; based on a local differential privacy technology, adopting an output noise maximum value algorithm to perform text confusion on original input data of a user; performing vector mapping on the confused text data through an embedding function; and performing encrypted transmission of the embedding vector to a server GPU, performing processing through a self-attention mechanism and a feedforward neural network, generating a prediction result, and returning the prediction result to the client. The application can solve the contradiction between data privacy protection and calculation efficiency in existing large model reasoning.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method and system for secure inference of large models that integrates trusted execution environment and differential privacy. Background Technology

[0002] In today's era, artificial intelligence is booming at an unprecedented pace, and the rise of large language models has ushered in a new era of intelligent interaction. At the end of 2022, OpenAI's ChatGPT attracted global attention, surpassing 100 million users in just two months. Since then, various large language models have sprung up like mushrooms after rain. Large Language Models (LLMs) refer to language models with billions or more parameters trained on massive amounts of text; hereinafter referred to as large models. With their powerful human-computer interaction and task reasoning capabilities, large models have been widely applied in various fields, including healthcare, education, law, and finance. These applications have brought huge economic benefits and demonstrated the enormous value potential of large models. However, with the diversification and complexity of application scenarios, large models face severe challenges in terms of security and privacy. Due to their high hardware resource requirements, large models are usually deployed on cloud servers, accessed by users through various interfaces. During interaction, large models need to process massive amounts of user data, which contains a large amount of sensitive personal information and trade secrets. Improper handling can easily lead to data security issues. Large-scale attacks in recent years have further exacerbated users' concerns about privacy protection.

[0003] To protect user data security, researchers have proposed various privacy protection technologies, including secure multi-party computation, differential privacy, and confidential computation. Among these, confidential computation protects data privacy through hardware-based Trusted Execution Environments (TEEs), offering better computational performance and practicality. TEEs, through processor-level security extensions such as ARM TrustZone and Intel SGX, create an independent execution zone outside the main operating system. This ensures that even if the host system or virtual machine is attacked, the runtime state within the execution zone, including CPU registers, memory, and sensitive I / O, remains confidential and intact. Furthermore, TEEs possess remote proof capabilities, allowing verification of their trustworthiness to third parties. However, TEEs are typically integrated on the CPU, which has resource limitations and is unsuitable for running large deep learning models. Therefore, researchers have designed delegation protocols to securely outsource some linear computations during inference to heterogeneous processors such as GPUs for acceleration. The TEE needs to convert linear layer data into an encrypted format, hand it over to the GPU for computation, and then decrypt the result back into the original input for nonlinear layers. To ensure the performance benefits of computational outsourcing, the encryption algorithm used cannot be overly complex.

[0004] Differential privacy (DP) is another technique used to enhance data privacy in large models during fine-tuning and inference. It's a privacy standard defined mathematically, representing an algorithm's properties rather than the characteristics of the data. Early differential privacy algorithms injected noise into statistical query results, making it impossible for attackers to accurately infer information about specific individuals in the dataset, even if they possessed the query results. Depending on the data processing location, differential privacy can be divided into centralized differential privacy (CDP) and local differential privacy (LDP). CDP requires uniform noise addition to the data on a central server, suitable for centralized data collection and processing scenarios. LDP, on the other hand, allows data owners to locally randomize and perturb the data before transmitting it to an untrusted data manager. In the local differential privacy model, the trustworthiness of the data manager is no longer a necessary condition, as the data received already conforms to the differential privacy standard. Compared to centralized differential privacy, local differential privacy has a significant advantage: data subjects do not need to trust any entity other than themselves. This advantage has led to the widespread application of LDP in practical systems. While local differential privacy methods can protect data privacy, they cannot guarantee the security of the noise addition process, and adding noise to all input text can impair the inference performance of large models.

[0005] In summary, Trusted Execution Environments (TEEs) and Differential Privacy (DPP) are two important privacy protection technologies, each playing a unique role in model inference and data processing. However, neither technology alone has its limitations. Therefore, researching a method for secure inference of large models that integrates TEEs and DPP is of great significance. This approach not only combines the advantages of both technologies to provide more comprehensive privacy protection but also effectively addresses increasingly complex security threats, offering a new solution for privacy computing in the era of large models. Summary of the Invention

[0006] To address the shortcomings of existing trusted execution environment-based methods, which suffer from significantly increased inference latency due to frequent data interactions and encryption / decryption operations, and differential privacy-based methods, which require access to model parameters and architecture and are difficult to apply to black-box service scenarios that require protection of model intellectual property, this invention proposes a secure inference method and system for large models that integrates trusted execution environment and differential privacy. This method can resolve the contradiction between data privacy protection and computational efficiency in existing large model inference.

[0007] To achieve the above objectives, the technical solution of the present invention includes the following:

[0008] A secure inference method for large models that integrates trusted execution environments and differential privacy, applying a local client endpoint, the method comprising:

[0009] The large model is divided into an embedding layer and a core processing layer. The embedding layer is deployed in a trusted execution environment on a local client, and the core processing layer is deployed in a remote server equipped with a multi-GPU accelerator card. The embedding layer is used to map the input text into an embedding vector, and the core processing layer is used to obtain the prediction result of the input text based on the embedding vector.

[0010] After obfuscating the original text based on differential privacy, the embedding vector generated based on the embedding layer is sent to the remote server so that the remote server can use the core processing layer to obtain and return the prediction result of the embedding vector.

[0011] Based on the prediction results of the embedding vector, a local vocabulary mapping is performed to obtain the inference results of the original text.

[0012] Furthermore, the obfuscation of the original text based on differential privacy includes:

[0013] The words in the original text are divided into sensitive words and non-sensitive words;

[0014] For each word w in the original text, the k nearest neighbors w′ of the word w are retrieved in the word embedding space using the cosine similarity metric. The cosine similarity between word w and its neighbors w′ is then normalized using a linear transformation to obtain the cosine similarity.norm (w,w′);

[0015] In the similarity cos norm Add Laplace noise of strength ∈ to (w,w′) to generate perturbation scores for neighboring words w′;

[0016] Based on the perturbation score, a neighboring word w′ is selected as a candidate word for the word w;

[0017] If word w is a sensitive word, use the candidate word to replace word w;

[0018] If the word w is a non-sensitive word, replace word w with the candidate word with probability p, and keep word w unchanged with probability 1-p.

[0019] Furthermore, the similarity Where cos(w,w′) represents the cosine similarity between the word w and its neighboring word w′, cos max =maxcos(w,w′),cos min =mincos(w,w′).

[0020] Furthermore, the core processing layer is used to obtain and return the prediction result of the embedded vector, including:

[0021] The embedded vector is processed by a multi-layer decoder network to obtain the output o of the multi-layer decoder network; wherein the decoder in the multi-layer decoder network adopts a self-attention mechanism and a feedforward neural network structure;

[0022] The output o is mapped to the vocabulary space using a linear layer SoftMax function to calculate the probability distribution of each candidate word;

[0023] The word with the highest probability is selected as the prediction result for the embedding vector.

[0024] Furthermore, the loss function L for training the large model total =L task +λ·L emo Among them, the task loss function T represents the number of words in the actual corpus, w t Let represent the embedding vector of the t-th word, and p represent the model's predicted generation of w given the first t-1 words of the input. t The probability, w <t This represents the first t-1 words of the input, where λ is a hyperparameter used to balance the weights of different loss functions, and the EMO loss function is used. p t e represents the probability that the model predicts will generate the next word.t e represents the word embedding vector corresponding to the next word predicted by the large model. j This represents the word embedding vector corresponding to the next word in the real corpus.

[0025] Furthermore, the feature is that the communication between the embedded layer and the core processing layer uses the AES-GCM encryption protocol to transmit data.

[0026] A large-scale model secure inference system that integrates trusted execution environment and differential privacy, the system comprising:

[0027] The model deployment module is used to divide a large model into an embedding layer and a core processing layer, and deploy the embedding layer in a trusted execution environment on a local client, and deploy the core processing layer in a remote server equipped with a multi-GPU accelerator card; wherein, the embedding layer is used to map the input text into an embedding vector, and the core processing layer is used to obtain the prediction result of the input text based on the embedding vector;

[0028] The embedding vector generation module is used to obfuscate the original text based on differential privacy, and then send the embedding vector generated based on the embedding layer to a remote server so that the remote server can use the core processing layer to obtain and return the prediction result of the embedding vector.

[0029] The reasoning result acquisition module is used to perform local vocabulary mapping based on the prediction result of the embedding vector to obtain the reasoning result of the original text.

[0030] An electronic device, characterized in that the electronic device comprises: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the large-model secure inference method that integrates trusted execution environment and differential privacy as described above.

[0031] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the large-model secure inference method that integrates trusted execution environment and differential privacy as described above.

[0032] A computer program product, characterized in that, when the computer program product is run on a computer device, the computer device executes the large-model secure inference method that integrates trusted execution environment and differential privacy as described above.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects.

[0034] 1. This invention addresses the latency issue caused by frequent encrypted interactions due to model segmentation in traditional TEE methods. It proposes a distributed deployment architecture for the embedding layer and the core processing layer, completely confining the sensitive text mapping process within a local SGX trusted environment. Simultaneously, it achieves efficient communication between the client and the GPU server via PyTorch RPC. Experiments on Llama series models verify that this strategy effectively reduces the latency of secure inference for large models.

[0035] 2. To reduce the impact of text obfuscation mechanisms on model performance, this invention designs a text obfuscation mechanism based on word frequency hierarchical protection: low-frequency sensitive words are automatically identified from the corpus, and a semantically similar candidate set is generated by combining cosine similarity retrieval in the word embedding space. Laplace noise perturbation is then added to the similarity to achieve differentiated replacement. This replacement strategy ensures the privacy of user input while maintaining the semantic coherence of the text before and after replacement as much as possible.

[0036] 3. This invention addresses the performance degradation caused by noise perturbations by utilizing a hybrid loss function to enhance the model's semantic understanding of obfuscated text during the fine-tuning stage of downstream tasks. The feasibility of this method has been verified in various downstream tasks, including text classification and dialogue summarization.

[0037] In summary, this invention significantly reduces the frequency of data interaction between TEE and GPU by optimizing the model deployment architecture, thereby reducing inference latency caused by encryption and decryption operations. Simultaneously, it achieves effective protection of user input data without requiring access to the complete parameters and architecture of the model, making it suitable for the black-box service model adopted by large real-world models. Furthermore, users can modify the differential privacy budget according to different application scenarios to balance privacy protection strength and model performance. This invention has good practicality and adaptability. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the large-scale model reasoning process of the method of this invention.

[0039] Figure 2 This is a schematic diagram of the distributed deployment of the model proposed by the method of this invention.

[0040] Figure 3 This is a schematic diagram of the text obfuscation mechanism proposed in this invention.

[0041] Figure 4 This is a comparison diagram of the reasoning time decomposition of the present invention and the prior art. Detailed Implementation

[0042] To better illustrate the large-model secure inference method that integrates trusted execution environment and differential privacy proposed in this invention, the following description, in conjunction with the accompanying drawings and specific implementation methods, takes the application of the Llama2-7B-chat model on the text classification dataset SST-2 as an example to further explain this invention.

[0043] Figure 1 This is a flowchart of the present invention, which is mainly divided into five parts: distributed deployment of the model, text obfuscation processing, embedding vector mapping, large model decoding output, and model fine-tuning.

[0044] Step 1: Distributed deployment of the model.

[0045] Based on the computational characteristics of large models and the need for user privacy protection, this invention divides large models into two key components: an embedding layer and a core processing layer. The main responsibility of the embedding layer is to map the input text data into a unified vector space for subsequent model processing. Figure 2 As shown, the embedding layer is deployed in the local client's SGX2 trusted environment, ensuring the privacy and integrity of user input data through hardware-level security isolation, while also ensuring the confidentiality of the embedding layer on the client side. On the other hand, the core processing layer of the large model undertakes more complex computational tasks and is deployed on a remote server equipped with multi-GPU accelerator cards to fully leverage the significant advantages of GPUs in parallel computing. To achieve efficient communication between the embedding layer and the core processing layer, this invention employs the Remote Procedure Call (RPC) interface from the PyTorch framework. This layered deployment strategy retains the parallel computing advantages of GPUs while using physical isolation mechanisms to confine sensitive data processing tasks to a secure local area.

[0046] Step 2: Text obfuscation.

[0047] This invention utilizes local differential privacy technology to obfuscate the original input data, thereby protecting sensitive user data. Specifically, while distributed model deployment eliminates the need for users to send plaintext input data to the server, the output of the embedding layer can still potentially leak private data. Attackers can launch inversion attacks using intermediate layer features, i.e., the embedding vectors. This paper proposes a text obfuscation mechanism based on an output noise maximization algorithm, such as... Figure 3 As shown, this mechanism constructs a hierarchical protection strategy through word frequency analysis: words with frequencies below a threshold θ are classified as sensitive words, while the rest are classified as non-sensitive words. Stop words are all non-sensitive words. Assuming the user dictionary is V, the word frequency distribution frequency(x) in a certain corpus is statistically analyzed. x∈V A sensitive word set is constructed by setting a threshold of θ.

[0048] S = {x∈V|frequency(x)<θ}.

[0049] The set of non-sensitive words is N = VS, mainly containing high-frequency words and stop words. Due to their high semantic redundancy and weak privacy relevance, a lenient processing strategy is adopted. The implementation of the text obfuscation mechanism relies on the following two key functions:

[0050] • Candidate word generation function: targeting the word w i This function uses cosine similarity to retrieve the k nearest neighbors (excluding the target word itself) in the word embedding space and employs normalization in the similarity calculation. First, the standard cosine similarity is calculated:

[0051]

[0052] Subsequently, a linear transformation is applied to normalize the similarity values ​​in the candidate set:

[0053]

[0054] Among them, cos max =maxcos(w,w j (1≤j≤k) and cos min =mincos(w,w) j (1≤j≤k) represent the maximum and minimum similarity values ​​in the candidate word set, respectively.

[0055] • Random mapping function: This function defines a probability function P to implement a differentiated processing strategy. For sensitive words, Laplace noise with intensity ∈ is added to generate a perturbation score, and the candidate word with the highest score after noise correction is selected for replacement.

[0056] score(w,w j )=cos norm (w,w j )+z,

[0057]

[0058] For non-sensitive words: execute the above text obfuscation strategy with probability p, and keep it unchanged with probability 1-p.

[0059] Step 3: Embedded vector mapping.

[0060] Obfuscated user input data X = <w′1,w′2,…,w′ nThe data is first sent to an embedding function deployed in a local trusted environment, where a vector mapping operation is performed. The mapping process generates fixed-dimensional vector representations by multiplying the embedding matrix with the input data. These vector representations preserve the semantic information of the original input while hiding the specific content. The calculation formula is as follows:

[0061] h = f emb (X)=W emb X,

[0062] Where h represents the generated embedding vector, W emb Let E(·) denote the embedding matrix, and let E(·) denote the embedding function.

[0063] Step 4: Decode and output the large model.

[0064] After receiving the encrypted embedding vector, the server performs the following operations to encrypt and transmit the embedding vector to the server's GPU.

[0065] Decoding Phase: The embedding vector h is first processed through a multi-layer decoder network. The decoder employs a self-attention mechanism and a feedforward neural network structure, as expressed in the formula:

[0066] h′=Decoder(h)=MultiHead(LayerNorm(h))+h,

[0067] o=FFN(LayerNorm(h′))+h′,

[0068] Here, MultiHead represents a multi-head self-attention layer, LayerNorm represents a layer normalization operation, and FFN represents a feedforward neural network.

[0069] Output layer processing: The decoder's output o is mapped to the vocabulary space through a linear layer SoftMax function, and the probability distribution of each candidate word is calculated:

[0070] p(w n+1 |w′1,w′2,…,w′ n =SoftMax(W o o+b o ),

[0071] Among them, W o and b o These are the weight matrix and bias vector of the output layer. Finally, the word with the highest probability is selected as the prediction result based on the probability distribution.

[0072]

[0073] The server returns the predicted next word ID to the client, which then performs local word mapping to convert the ID into the actual word.

[0074] Step 5: Model fine-tuning.

[0075] A hybrid loss function is calculated to efficiently fine-tune the parameters of a large model on downstream task datasets. To reduce the impact of text obfuscation mechanisms on model performance, the model can be fine-tuned on the dataset in a small number of cycles. An EMO loss function is introduced on top of the cross-entropy loss function to update the model's gradients. The hybrid loss function is calculated as follows:

[0076] L total =L task +λ·L emo ,

[0077] Here, λ is a hyperparameter used to balance the weights of different loss functions.

[0078] The formula for calculating the task loss function is:

[0079]

[0080] Where T represents the number of words in the real corpus, w t Let represent the embedding vector of the t-th word, and p represent the model's predicted generation of w given the first t-1 words of the input. t The probability, w <t This represents the first t-1 words of the input.

[0081] The EMO loss function is calculated using the following formula:

[0082]

[0083] Where, p t It is the probability of generating the next word predicted by the model, e t This represents the word embedding vector corresponding to the next word predicted by the model, and both are calculated through the model's forward propagation process. j The word embedding vector representing the real word is usually obtained through the embedding layer mapping of a large model.

[0084] The gradient update formula during training is:

[0085]

[0086] Where η is the learning rate. It refers to the gradient of the model parameter θ.

[0087] Fine-tuning using a hybrid loss function can enhance the model's ability to generalize to obfuscated text. The model can better understand and process inputs that have been obfuscated, thereby maintaining high prediction performance while ensuring privacy and security.

[0088] This solution retains the parallel computing advantages of GPUs while protecting sensitive user data through the physical isolation mechanism and differential privacy of TEE. It maintains high inference performance while ensuring privacy and security, effectively balancing the relationship between security and efficiency.

[0089] The method of this invention will be further explained below using the Llama2-7B-chat model and the SST-2 dataset as examples.

[0090] Step 1. Distributed Model Deployment. The Llama2-7B-chat model is divided into two key parts: the embedding layer is deployed on a local client equipped with an Intel SGX2 processor, and the core processing layer is deployed on a remote server equipped with two NVIDIA A100 GPUs. The embedding layer contains an embedding matrix with 32,000 labeled elements and a dimension of 4,096; the core processing layer contains 32 Transformer decoders, each with 32 attention heads. The two parts establish a secure communication channel via PyTorch's RPC interface.

[0091] Step 2. Text Obfuscation. Taking the SST-2 dataset as an example, firstly, the word frequency distribution is statistically analyzed, and a threshold θ = 0.2 is set (i.e., words with a frequency below 20% are considered sensitive words). For the identified sensitive words, Laplace noise intensity ∈ = 0.7 is applied for perturbation; for non-sensitive words, the replacement probability p = 0.3 is set. Candidate word selection selects k = 20 most similar words, calculates cosine similarity, and performs normalization. For example, the user input text: "we never originallyfeel involved with the movie, as all della its tips remain just that:abstractopinions", after text obfuscation, becomes: "we never really feel involved with the story, as all of its ideas remain just that:abstract ideas". In this text, "originally" is replaced with "really", "movie" is replaced with "story", "della" is replaced with "of", "tips" is replaced with "ideas", and "opinions" is replaced with "ideas". These replacements preserve the basic semantic structure of the original text, but reduce the personalized features and sensitive information in the text.

[0092] Step 3. Embedded Vector Mapping. The obfuscated text is combined with the prompt template "Classify the sentiment of the given movie review text into Positive or Negative. Movie Review:", and mapped to a 4,096-dimensional vector using an embedding function. The computation is performed within an SGX2 trusted environment. After the embedding computation is complete, the vector is encrypted using the AES-GCM-256 algorithm, and the key is securely exchanged via the SGX2 remote authentication mechanism.

[0093] Different templates can be used depending on the downstream tasks of the large model. Concatenating the obfuscated text and the prompt template is to improve the model's performance on specific tasks.

[0094] Step 4. Decoding Output. The server GPU receives the encrypted embedding vector, decrypts it, and processes it using a Transformer decoder. For the SST-2 sentiment classification task, the maximum decoding length is set to 10, and the temperature parameter is set to 0.1 to enhance deterministic output. The decoder output is processed through a softmax layer, and the word sequence with the highest probability is selected and returned to the client via a token ID. The client then uses its local vocabulary mapping to reconstruct the actual words, and the final output is "Negative," indicating that the obfuscated text is classified as negative sentiment.

[0095] Step 5. Model Fine-tuning. To reduce the impact of text obfuscation on performance, efficient parameter fine-tuning was performed on the SST-2 training set. A hybrid loss function was used, with the weight λ set to 0.5, and the cross-entropy loss and EMO loss fused in a 1:1 ratio. The AdamW optimizer was used with a learning rate of 3e-5 and a weight decay of 0.01, fine-tuning for 3 epochs. LoRA low-rank adaptation was employed, with the rank set to 8, updating only the parameters of the model's attention module while freezing the remaining parameters to conserve computational resources. Experimental results show that at ε = 2, the method of this invention achieves 90% accuracy on the SST-2 test set, only 7% lower than the original model. Furthermore, compared to methods entirely based on TEE, inference latency is significantly reduced.

[0096] Finally, to verify the effectiveness of the proposed invention, this experiment selected SANTEXT+ (Yue X, Du M, Wang T, et al. Differential privacy for text analytics via natural text sanitization[C] / / Zong C, Xia F, Li W, et al. Findings of ACL:ACL / IJCNLP 2021Findings of the Association for Computational Linguistics:ACL / IJCNLP 2021,Online Event,August 1-6,2021.Association for Computational Linguistics,2021:3853-3866.) and CusText+ (Chen S, Mo F, Wang Y, et al.A customized text sanitization mechanism with differential privacy[C] / / Rogers A,Boyd-Graber JL,Okazaki N.Findings of the Association for Computational Linguistics:ACL (2023, Toronto, Canada, July 9-14, 2023. Association for Computational Linguistics, 2023: 5747-5758.) This model was used as a baseline for comparison, and tests were conducted on the text classification task datasets SST-2 and QNLI based on the bert-base-uncased model. The following is a brief introduction to these two baseline models:

[0097] • SANTEXT+: This method divides vocabulary into sensitive sets. With non-sensitive sets A utility-optimized local differential privacy (Metric LDP) framework was constructed. This framework can adaptively allocate the privacy budget based on semantic distance. During the pre-training phase, the model uses perturbed text for mask prediction, thereby enhancing the model's robustness to noise.

[0098] • CusText+: This model uses a mapping function to assign a customizable output set to each input word. The size of this output set can be flexibly adjusted via the hyperparameter k to meet different privacy and practicality requirements. The model employs an exponential sampling function, selecting a token from the custom output set to replace the original token.

[0099] Table 1 lists the classification accuracy of different privacy protection mechanisms on the SST-2 and QNLI datasets under similar privacy protection levels, where k controls the candidate set size for each word. As the privacy budget ∈ increases, the BERT model shows a slight improvement in accuracy on SST-2 and QNLI due to reduced noise perturbation. Under different privacy budget conditions, the secure inference method proposed in this invention consistently outperforms other baseline methods.

[0100] Table 1. Comparison of accuracy rates of different privacy protection mechanisms

[0101]

[0102]

[0103] To evaluate the additional inference overhead introduced by this invention, this experiment uses two lightweight large models, Llama2-7B-chat and Llama3-8B-Instruct, and uses the average token generation time on the SST-2 dataset as the core evaluation metric. Inference efficiency is calculated through multiple measurements. The inference efficiency metric is based on the average forward propagation time, and the calculation formula is:

[0104]

[0105] in, Let M be the inference time for the j-th sample, and M be the total number of test samples, defaulting to 100. Because previous methods based on trusted execution environments had different preconditions, direct fair comparisons were difficult. Therefore, this paper establishes two evaluation baseline models:

[0106] • CPU+GPU Baseline: In this baseline setup, the client is responsible for executing the embedding layer of the large model in a regular computing environment, while the core layer of the model runs on a GPU server. No privacy protection measures are employed throughout the entire inference process.

[0107] • GPU baseline: All layers of the large model are deployed on GPU servers for execution, and similarly, no privacy protection mechanisms are used throughout the process.

[0108] like Figure 4As shown, the proposed method introduces additional time overhead of 0.7152 seconds (28.78%) and 0.7432 seconds (38.12%) on Llama2-7B and Llama3-8B models, respectively, compared to the CPU+GPU baseline. The main source of this overhead is the SGX2 encryption mechanism, which utilizes a hardware-level memory encryption engine to provide strong protection for data confidentiality and integrity in secure areas. The time overhead of the text obfuscation mechanism, including candidate word generation and replacement, is relatively small, accounting for only 0.6% and 0.74% of the total inference time, respectively. Furthermore, since candidate words for different large models can be pre-computed and stored, the efficiency of this mechanism can be further improved in practical applications.

[0109] In summary, due to the limited hardware resources within the TEE, researchers have proposed a deep learning model segmentation strategy to improve inference efficiency while ensuring data privacy. This strategy offloads the linear computations of the model to the GPU for acceleration. Depending on the encryption method used for input features, these methods can be categorized into computational outsourcing methods based on One-Time Password (OTP) encryption and computational outsourcing methods based on linear transformations. Since nonlinear functions often involve complex operations such as exponential and logarithmic functions, current encryption schemes still struggle to support them efficiently. Furthermore, large models are typically composed of multiple layers of self-attention mechanisms, with each layer consisting of alternating linear and nonlinear computations. Therefore, when existing model segmentation methods are applied to large models, they lead to frequent data interactions between the TEE and the GPU. Each data exchange requires encryption and decryption operations, a mechanism crucial for data security but also significantly increasing inference latency.

[0110] Differential privacy-based model security inference methods, grounded in rigorous mathematical proofs, provide effective defense mechanisms for large models, resisting various privacy threats such as membership inference attacks, attribute inference attacks, and embedding inversion attacks. However, applying differential privacy techniques for privacy protection requires access to the model's parameters and architecture, which is difficult to achieve in real-world applications. Model parameters and architecture are often core assets of enterprises. Therefore, to protect the intellectual property of models, many deep learning models and large models provide services in a black-box manner, where users can only obtain the output results through input data.

[0111] Based on the above, this invention proposes a collaborative security framework that integrates a trusted execution environment and differential privacy. By splitting a large model into a layered deployment architecture of an embedding layer and a core processing layer, the embedding vector mapping of sensitive text is performed in the client's local SGX2 trusted environment. Hardware-level isolation is used to ensure the privacy of input data, while the computationally intensive core processing layer is deployed on a GPU server to maintain efficient parallel computing capabilities. The two layers achieve collaborative inference through encrypted communication.

[0112] Furthermore, to protect the privacy of user input data, this invention also designs a text obfuscation mechanism based on the maximum output noise algorithm. This mechanism applies Laplace noise perturbation to low-frequency sensitive words, combines cosine similarity retrieval in the word embedding space to generate semantically similar replacement words, and implements a probability preservation strategy for high-frequency non-sensitive words, thereby maximizing semantic coherence while protecting privacy.

[0113] Although the specific details, implementation algorithms, and accompanying drawings of the present invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. The invention should not be limited to the content disclosed in the preferred embodiments and accompanying drawings; the scope of protection claimed by the invention is defined by the claims.

Claims

1. A secure reasoning method for large models that integrates trusted execution environment and differential privacy, characterized in that, The method, which applies to a local client, includes: The large model is divided into an embedding layer and a core processing layer. The embedding layer is deployed in a trusted execution environment on a local client, and the core processing layer is deployed in a remote server equipped with a multi-GPU accelerator card. The embedding layer is used to map the input text into an embedding vector, and the core processing layer is used to obtain the prediction result of the input text based on the embedding vector. After obfuscating the original text based on differential privacy, the obfuscated original text is sent to the embedding layer deployed in a local trusted environment, and the embedding vector generated based on the embedding layer is sent to the remote server, so that the remote server can use the core processing layer to obtain and return the prediction result of the embedding vector. Based on the prediction results of the embedding vector, a local vocabulary mapping is performed to obtain the inference results of the original text; Among these, the obfuscation of the original text based on differential privacy includes: The words in the original text are divided into sensitive words and non-sensitive words; For each word in the original text Using cosine similarity as a metric, the nearest neighbors of the word in the word embedding space are retrieved. neighboring words and the word and neighboring words The cosine similarity is normalized using a linear transformation to obtain the similarity. ; The similarity Add strength to Laplace noise to generate neighboring words The perturbation score; Based on the perturbation score, select a neighboring word. As the word Candidate words; In the word In the case of a sensitive word, use the candidate word to replace the word. ; In the word In the case of non-sensitive words, based on probability Use candidate words to replace words and with probability Keep the word constant.

2. The method according to claim 1, characterized in that, The similarity ;in, This word and neighboring words cosine similarity, , .

3. The method according to claim 1, characterized in that, The core processing layer is used to obtain and return the prediction result of the embedded vector, including: The embedded vector is processed by a multi-layer decoder network to obtain the output of the multi-layer decoder network. The decoder in the multi-layer decoder network employs a self-attention mechanism and a feedforward neural network structure. The output is processed by the linear layer SoftMax function. Mapping to the vocabulary space to compute the probability distribution of each candidate word; The word with the highest probability is selected as the prediction result for the embedding vector.

4. The method according to claim 1, characterized in that, Loss function for training the large model Among them, the task loss function , This indicates the number of words in the actual corpus. Indicates the first The embedding vector of each word, Indicates before the given input Given a number of words, the model predicts the generation. The probability, Indicates the preceding input One word, These are hyperparameters used to balance the weights of different loss functions; the EMO loss function. , This represents the probability that the model predicts the generation of the next word. This represents the word embedding vector corresponding to the next word predicted by the large model. This represents the word embedding vector corresponding to the next word in the real corpus.

5. The method according to any one of claims 1 to 4, characterized in that, Communication between the embedded layer and the core processing layer uses the AES-GCM encryption protocol to transmit data.

6. A large-scale model secure inference system integrating trusted execution environment and differential privacy, characterized in that, The system includes: The model deployment module is used to divide a large model into an embedding layer and a core processing layer, and deploy the embedding layer in a trusted execution environment on a local client, and deploy the core processing layer in a remote server equipped with a multi-GPU accelerator card; wherein, the embedding layer is used to map the input text into an embedding vector, and the core processing layer is used to obtain the prediction result of the input text based on the embedding vector; The embedding vector generation module is used to obfuscate the original text based on differential privacy, send the obfuscated original text to the embedding layer deployed in a local trusted environment, and generate the embedding vector based on the embedding layer to the remote server, so that the remote server can use the core processing layer to obtain and return the prediction result of the embedding vector. The reasoning result acquisition module is used to perform local vocabulary mapping based on the prediction result of the embedding vector to obtain the reasoning result of the original text; Among these, the obfuscation of the original text based on differential privacy includes: The words in the original text are divided into sensitive words and non-sensitive words; For each word in the original text Using cosine similarity as a metric, the nearest neighbors of the word in the word embedding space are retrieved. neighboring words and the word and neighboring words The cosine similarity is normalized using a linear transformation to obtain the similarity. ; The similarity Add strength to Laplace noise to generate neighboring words The perturbation score; Based on the perturbation score, select a neighboring word. As the word Candidate words; In the word In the case of a sensitive word, use the candidate word to replace the word. ; In the word In the case of non-sensitive words, based on probability Use candidate words to replace words and with probability Keep the word constant.

7. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the large-model secure inference method that integrates trusted execution environment and differential privacy as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the large-model secure inference method that integrates trusted execution environment and differential privacy as described in any one of claims 1-5.

9. A computer program product, characterized in that, When the computer program product is run on a computer device, the computer device performs the large-model secure inference method that integrates trusted execution environment and differential privacy as described in any one of claims 1-5.

Images