Edge side large model security reasoning method based on trusted execution environment
By adopting the method of efficient fine-tuning vector protection module and random adaptation head combination module on edge devices, the problems of large model inference latency and memory requirements are solved, efficient and secure model inference is achieved, model theft attacks are resisted, and model performance is maintained.
Patent Information
- Application Number
- CN202510756515.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-09
- Publication Date
- 2025-09-19
AI Technical Summary
Existing secure inference methods introduce massive secure memory requirements and complex calculations when protecting large models, resulting in increased inference latency and the risk of sensitive parameter leakage, making them unable to effectively defend against model theft attacks.
It adopts an efficient fine-tuning vector protection module and a random adaptation head combination module to protect sensitive parameters in a trusted execution environment, accelerate inference in the REE of edge devices, utilize resources such as GPU and NPU, and combine random matrix transformation to prevent model theft attacks.
Effectively reduce secure memory requirements, improve inference speed, reduce latency, maintain model accuracy, and effectively defend against model theft attacks.
Smart Images
Figure CN120671830A_ABST
Abstract
Description
(1) Technical field:
[0001] The present application relates to the field of secure reasoning technology, and is concerned with a large-scale secure reasoning method for edge-side models based on a trusted execution environment. (2) Technical background:
[0002] Large models based on the Transformer architecture have achieved tremendous success in fields such as natural language processing, computer vision, and time series data analysis. To mitigate the degradation of service quality caused by network latency and the upload of user privacy data, a growing number of model vendors are deploying large models on users' edge devices. However, edge device environments are not controlled by the model provider, and deployed models can be stolen by users or potential adversaries. Training large models often consumes significant computing resources and time, and relies on high-quality private datasets from the model vendor. Highly trained models are crucial for vendors to maintain their market competitiveness. Therefore, model theft is a serious loss of digital assets and can even threaten a company's survival.
[0003] In recent years, secure reasoning based on a Trusted Execution Environment (TEE) has seen significant development. These methods protect all or part of a model's key parameters within a TEE, preventing adversaries from gaining access to the complete model parameters and thus ensuring model confidentiality. Furthermore, some advanced methods employ various parameter obfuscation mechanisms to offload obfuscated model parameters to a Rich Execution Environment (REE) for acceleration, thereby ensuring real-time secure reasoning.
[0004] However, existing secure inference methods focus on protecting small models. When protecting large models with huge parameter counts, they introduce huge secure memory requirements and complex calculations, which seriously damages the real-time performance of inference. Generally speaking, Intel SGX's secure memory is often limited to 128MB. When the memory requirements are exceeded, the cryptographic protection mechanism will introduce a large amount of additional calculations, which will lead to increased inference latency. Test results on the RoBERTa large model show that the single inference latency in TEE is as high as 7.73 seconds, which is 53.60 times higher than REE inference. As shown in the attached figure, Figure 2 As shown, this seriously affects the user experience. Furthermore, existing methods obfuscate sensitive parameters or place them plaintext in REEs to accelerate inference, significantly exacerbating the risk of parameter leakage and failing to achieve the upper bound of no sensitive parameter leakage.
[0005] To address the aforementioned issues, this paper innovatively proposes a secure inference method for large models at the edge based on a trusted execution environment (TEE). The method consists of an efficient fine-tuning vector protection module and a random adaptation head combination module. First, during the training phase, the manufacturer selects a publicly available pre-trained model as the base and adds an adaptation head for each target fine-tuning module. Each adaptation head consists of a set of random low-rank matrices frozen during training and a set of fine-tuning vectors. As a result, all private information is confined to the fine-tuning vectors, while the base model and random matrices do not require protection. During the inference phase, the efficient fine-tuning vector protection module offloads the model base and the random low-rank matrices of the adaptation head to the REE for inference, and stores the fine-tuning vectors in the TEE for protection. During this process, the bulk of the inference operations (over 95%) are offloaded to the REE, enabling acceleration using computing resources such as GPUs and NPUs, improving inference performance. Private parameters are securely stored in the TEE, effectively preventing sensitive parameter leakage and ensuring model parameter confidentiality. Furthermore, the random adaptation head combination module combines different adaptation heads in real time during inference and updates the model parameters in the REE. The mapping processes of different random matrices are different, which can effectively change the distribution characteristics of the intermediate results of the model, thereby preventing the adversary from using the intermediate results to strengthen the training process and resist model theft attacks.
[0006] The present invention has the following advantages: First, the security reasoning solution proposed in the present invention can effectively reduce the demand for secure memory and is more suitable for edge devices with limited computing resources, such as the attached Figure 1 Second, the present invention can greatly speed up the safety reasoning of large models, as shown in the attached Figure 2 Third, the present invention will hardly cause the model accuracy to drop, and can maintain the excellent performance of large models, as shown in the attached Figure 3 Fourth, the present invention can effectively resist model stealing attacks and effectively reduce the accuracy of model stealing attacks, as shown in the attached Figure 4 shown. (3) Summary of the invention:
[0007] 1. Purpose:
[0008] The purpose of the present invention is to propose a secure reasoning method for large models on the edge side based on a trusted execution environment. It aims to address the problem that existing secure reasoning methods introduce huge amounts of secure memory and complex calculations, resulting in a sharp increase in model reasoning latency. Through the efficient fine-tuning vector protection module and random adaptation head combination module proposed in the present invention, the reasoning of large models on edge devices can be protected with high security and low latency, thereby protecting intellectual property and digital assets from infringement.
[0009] 2. Technical solution:
[0010] Main ideas: The main ideas of this plan are as follows Figure 5As shown, the process can be divided into the following steps: First, the model provider selects a public pre-trained model on the cloud server and fine-tunes the model using different random seeds based on private training data, generating multiple sets of adaptation headers. Each set of adaptation headers consists of a globally shared low-rank random matrix and a fine-tuning vector containing sensitive parameters. Second, after model training is complete, the model provider encrypts the multiple sets of adaptation headers using a secure transmission method supported by the Trusted Execution Environment (TEE) and securely sends them to the edge TEE for decryption. Third, the edge TEE randomly selects adaptation headers and combines them using a random adaptation head combination module to generate a combined adaptation header. Part of the random low-rank matrix of the combined adaptation header and the public pre-trained model are offloaded to the edge REE for inference. Fourth, during inference, inference data flows between the edge REE and the TEE. The inference portion involving the fine-tuning vector is performed within the TEE, while large-scale computations that require no protection are accelerated within the REE. Fifth, the TEE counts the total number of inferences. Whenever the cumulative number of inferences exceeds a threshold, step three is repeated to update the parameters within the REE, thereby defending against model theft attacks.
[0011] 3. Advantages and effects:
[0012] This paper provides a secure inference method for large models at the edge based on a trusted execution environment. It innovatively designs an efficient fine-tuning vector protection module to address the existing methods' reliance on massive amounts of secure memory. This method effectively ensures model confidentiality while maintaining high inference speeds. It also proposes a randomized adaptive header combination module to defend against model theft attacks. Experiments have verified the effectiveness of this method. (4) Description of the accompanying drawings:
[0013] Figure 1 This is a comparison chart of the memory requirements of this solution and existing security reasoning solutions.
[0014] Figure 2 This is a comparison chart of the inference latency between this solution and the existing security inference solution.
[0015] Figure 3 This is a comparison chart of the accuracy of this solution and the existing solution.
[0016] Figure 4 This is the effect diagram of this solution against model stealing attacks.
[0017] Figure 5 This is the safety reasoning flow chart for this solution. (V) Specific implementation methods:
[0018] The specific implementation steps of this solution are as follows:
[0019] Step 1: Train efficient fine-tuning vectors
[0020] At present, pre-trained large models have achieved excellent results on a large number of tasks and can effectively reduce the training time on downstream tasks. Most model manufacturers fine-tune based on open source pre-trained large models and use their own private data sets to adapt to professional field tasks. Based on the Parameter Efficient Fine-Tune (PEFT) theory, the fine-tuning performance of large models is closely related to the very small number of parameters of their intrinsic dimensions, that is, the parameter increment can be expressed as a low-rank matrix. Inspired by VeRA, the present invention randomly initializes low-rank A and B matrices, and trains multiple groups of efficient fine-tuning vectors, thereby controlling all sensitive parameters related to privacy training data in a very small number of fine-tuning vectors, which facilitates the subsequent steps to be included in the TEE for protection with low memory overhead and computational overhead. The training process is shown as follows:
[0021]
[0022] Where n represents the number of adaptation heads, b i with d i is the fine-tuning vector, A i With B i is a random low-rank adaptation matrix, W is the pre-training parameter weight, W′ i is the fine-tuned parameter of the i-th group of adaptation heads. During training, each group of adaptation heads uses a different random number seed to ensure zero repetition of the random low-rank adaptation matrix. The parameter increments are not incorporated into the model weights, but are included in the TEE for protection in subsequent steps.
[0023] Step 2: Safely distribute the model
[0024] The training in step 1 is completed on a server controlled by the model provider. To ensure that the model parameters are not stolen by the enemy during the transmission process, the cloud server directly connects to the edge device TEE, encrypts the key parameters in the adaptation header, and decrypts them in the edge TEE. If the edge device needs to store the adaptation header parameters in non-secure memory, it must be securely stored using a key that is only available to the TEE. In addition, since the parameters of the large pre-trained model on the base, which occupies more than 99% of the volume, are publicly available, there is no need to encrypt them and they can be transmitted directly to the edge device REE in plain text, thereby improving transmission efficiency.
[0025] Step 3: Random Adaptation Head Combination
[0026] During model inference, an adversary can record the intermediate results of model inference and match them during a model stealing attack, thereby enhancing the effectiveness of the attack. To defend against model stealing attacks, this paper proposes a random adaptive head combination module. Based on the associative property of linear operators, multiple sets of adaptive heads are linearly combined. During the inference process, the protected model parameters are changed in real time to alter the distribution of the intermediate inference results of the large model, preventing the adversary from using the intermediate results to enhance the model stealing attack. The process can be described as follows:
[0027]
[0028] Where T is the number of adaptation heads selected for each combination, t i is the combined coefficient of the i-th group of adaptation heads. Due to the mapping relationship between different random matrices, the distribution of intermediate results will change, preventing adversaries from leveraging intermediate results to enhance model theft attacks. This approach effectively avoids the drawbacks of existing methods that rely on OTP for intermediate result protection, reduces the computational complexity within the TEE, and offloads computationally intensive activation functions like softmax to the REE, further improving computational efficiency and inference speed.
[0029] Step 4: Efficiently fine-tune vector protection inference
[0030] Since the random low-rank matrix does not contain any training information, the present invention offloads the globally shared random low-rank matrix A to the REE side for calculation, and incorporates the random low-rank matrix B and fine-tuning vectors b, d into the TEE for protection reasoning. Since the random low-rank matrices A and B are shared by all fine-tuning vectors, the present invention can reuse the matrix in the TEE and only adapt the corresponding fine-tuning vectors b and d at different parameter layers, thereby maximizing the reduction of the security memory requirements required for large model security reasoning and avoiding the introduction of O(n 3 ) complex calculations, significantly improving the reasoning speed. The process can be formally described as follows:
[0031]
[0032] Among them, the random low-rank matrix A of the efficient fine-tuning module does not contain any private information, so it is offloaded to the REE for inference. Similarly, the complex operations of the pre-trained model are accelerated in the REE. Only low-complexity bBd low-rank matrix operations and vector operations are performed in the TEE, which can effectively reduce the computational complexity to O(n 2 ) and greatly reduces secure memory reliance. The results of operations in the TEE are passed to the REE for merging, and the next operator inference continues until the inference is complete.
[0033] During the inference process, the trusted program in the TEE counts the number of inferences of the current parameters in real time. When the designed threshold is reached, step 3 is re-executed to update the protected parameters in the TEE and the random low-rank matrix A in the REE, perturbing the distribution of intermediate results and thus defending against model theft attacks.
Claims
1. A secure reasoning method for large models at the edge based on a trusted execution environment (TEE). This method significantly reduces the demand for secure memory and computational complexity for secure reasoning of large models, and can significantly accelerate reasoning without leaking any sensitive information.
2. The efficient fine-tuning vector protection module as described in claim 1 utilizes a trusted execution environment to protect the fine-tuning vector inference process based on a random low-rank adaptive matrix, which can greatly reduce the security memory requirements and computational complexity while ensuring that no sensitive parameters are leaked, thereby improving the inference speed.
3. The random adaptive head combination module as described in claim 1 changes the parameter information protected in the trusted execution environment in real time based on linear combination, changing the distribution characteristics of intermediate results, thereby resisting model theft attacks and protecting the model as black box reasoning.