LLM Inference Encryption for Cloud Data Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of large language models on cloud services exposes user data to privacy risks through plaintext interactions, including transmission risks, operator theft, tenant snooping, and memory privacy leakage, with existing solutions like homomorphic cryptography being slow and limited to small models, and differential privacy only protecting unidirectional data flow.
Innovation Solution
An inference method and apparatus for large language models that perform encryption on input text to generate ciphertext, use an encrypted model for inference, and decrypt the result ciphertext back to plaintext, ensuring bidirectional data protection through transverse and longitudinal jitter encryption techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If homomorphic cryptography is used to protect data privacy, then data security is improved, but inference speed deteriorates and model size is limited
Solution Approach 1:
The patent segments the encryption approach into two parts: homomorphic encryption is applied selectively only to the input data, while the model parameters and computation process use conventional methods. This segmentation allows maintaining data security for inputs without suffering from the full performance penalty of homomorphic encryption throughout the entire inference process.
Solution Approach 2:
The patent introduces an intermediary approach by using homomorphic encryption only for the input data layer, acting as a mediator between complete encryption (secure but slow) and no encryption (fast but insecure). This partial application of homomorphic encryption provides security where needed while maintaining overall system performance.
2Reliability
If homomorphic cryptography is used to protect data privacy, then data security is improved, but applicable model size deteriorates (limited to small models only)
Solution Approach 1:
The patent segments the encryption scope to apply homomorphic encryption only to input data rather than the entire model, enabling the approach to work with large-scale models that would be incompatible with full homomorphic encryption solutions.
Solution Approach 2:
The patent changes the parameter of encryption application from 'all model components' to 'input data only', transforming the computational burden from intractable for large models to manageable, thus extending applicability to large-scale language models.
3Reliability
If differential privacy is used to protect data communication, then unidirectional data protection (client to server) is improved, but bidirectional protection deteriorates
Solution Approach 1:
The patent inverts the conventional differential privacy approach by applying encryption to the input data before it reaches the server, rather than adding noise to protect outputs. This inversion enables protection of both input and output data, achieving bidirectional security.
Solution Approach 2:
The patent uses homomorphic encryption as an intermediary mechanism that enables bidirectional protection: it protects input data during transmission to the server and also protects the output results during return to the client, overcoming the unidirectional limitation of differential privacy.
4Ease of operation
If plaintext interaction is used for cloud service, then ease of operation is improved, but privacy leakage risk worsens
Solution Approach 1:
The patent applies preliminary encryption to the input data before it leaves the client device. This preliminary action ensures that sensitive data is protected before entering the cloud environment, maintaining ease of operation while eliminating privacy leakage risks associated with plaintext transmission and storage.
Data Source
AI summary
Provided are an inference method and apparatus for a large language model, a device, and a storage medium. The inference method for the large language model includes: performing encryption on a target input text to obtain a target input ciphertext; sending the target input ciphertext to a server so that an encrypted model is used for performing inference on the target input ciphertext by the server to obtain a target result ciphertext; receiving the target result ciphertext sent by the server; and performing decryption on the target result ciphertext to obtain a target result plaintext.