AI Model Execution With GPU Memory Isolation and TEE Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing use of artificial intelligence (AI) and machine learning (ML) models in various applications poses challenges in protecting the confidentiality and integrity of these models during execution, particularly on GPUs, where they are vulnerable to potential leaks of proprietary data and model information.
Innovation Solution
The implementation of direct-to-accelerator security technologies and policies to protect AI/ML models and inferencing data during operation on GPUs or other accelerators, ensuring secure I/O operations and data isolation through mechanisms like Trusted Execution Environment (TEE)-I/O support and secure virtual machine environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI/ML models are executed on GPUs for high-performance computing, then processing speed and productivity are improved, but vulnerability to data leaks and security breaches increases
Solution Approach 1:
The patent segments the GPU execution environment into isolated secure regions using memory encryption and secure enclaves. Model weights, biases, and intermediate computations are partitioned into protected memory segments that are encrypted and isolated from unauthorized access, allowing high-performance processing while preventing data exfiltration through memory interception.
Solution Approach 2:
The patent introduces cryptographic intermediaries including hardware-based secure enclaves and encrypted memory buffers between the GPU computing core and external memory systems. These intermediaries act as security layers that allow data to flow for processing while preventing direct access to model parameters, thus maintaining both processing speed and security.
2Reliability
If traditional security measures are implemented to protect model confidentiality, then security is improved, but processing performance and productivity deteriorate
Solution Approach 1:
The patent replaces traditional software-based security mechanisms with hardware-enforced security features including cryptographic accelerators, hardware-based memory encryption, and secure enclave processors. These hardware-based solutions provide strong security guarantees without introducing the overhead and performance penalties associated with software security layers, thus maintaining high processing performance while ensuring model confidentiality.
3Loss of information
If model parameters are encrypted to prevent leaks, then confidentiality is improved, but access speed and processing efficiency worsen
Solution Approach 1:
The patent implements preliminary encryption of model parameters and computation intermediates before they are written to memory or transmitted between GPU cores. Cryptographic operations are performed in advance during data preparation phases, allowing the encrypted data to be stored and processed without real-time decryption overhead. This preliminary action ensures confidentiality while minimizing impact on access speed during the main computation phases.
Data Source
AI summary
Preservation of confidentiality and integrity of models during execution is described. An example of an apparatus includes a memory to store data for processing of one or more artificial intelligence (AI) models for an application; and processing resources including a graphics processing unit (GPU); a trusted execution environment (TEE); and circuitry to protect confidentiality and integrity of AI models during execution. The circuitry includes circuitry to obtain an AI model from a model server, the AI model being encrypted; circuitry to obtain a model license and a model encryption key for the AI model; circuitry to decrypt the AI model utilizing the model encryption key, and to re-encrypt the AI model with a second encryption key; and circuitry to perform an inference process for the AI model to generate inference output data for the application.