Partitioned Convolution Execution in TEE-Limited Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models, especially large-scale models, require significant computing and memory resources, and executing them in untrusted environments poses privacy risks, while local execution on devices with limited resources leads to accuracy and latency issues.
Innovation Solution
Execute deep learning models in a trusted execution environment (TEE) by partitioning convolutions into multiple sequential operations, storing input portions in a protected memory area, and performing single matrix multiplications for each partition, combining results to achieve the convolution output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep learning models are executed in untrusted environments, then computing speed is improved, but data security deteriorates
Solution Approach 1:
The patent divides the convolution operation into multiple partitioned convolutions, where each partition is processed separately within the TEE. This segmentation allows the model to execute securely in the trusted environment while maintaining computational efficiency through systematic processing of divided data chunks.
2Reliability
If deep learning models are executed locally on devices with limited resources, then data security is improved, but computing speed and model accuracy deteriorate
Solution Approach 1:
The patent introduces a trusted execution environment as an intermediary between the client device and the deep learning model execution. The TEE acts as a secure mediator that enables remote model execution while guaranteeing data confidentiality and integrity, thus resolving the contradiction between local execution security and remote execution performance.
3Productivity
If the protected memory area in TEE is increased to store more input data, then processing capability is improved, but storage efficiency deteriorates
Solution Approach 1:
The patent segments the input data into multiple input portions, where each portion is stored and processed separately in the protected memory area. This segmentation enables the system to handle large-scale models with limited TEE memory by processing data in manageable chunks, thus improving processing capability without requiring proportional increases in protected storage capacity.
Solution Approach 2:
The patent employs periodic action by sequentially loading different input portions into the protected memory area for processing. Each input portion is loaded, processed through matrix multiplication, and then removed before the next portion is loaded. This periodic load-process-remove cycle enables efficient utilization of limited TEE memory resources while maintaining high processing capability.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
In accordance with implementations of the subject matter described herein, there is provided a solution for execution of a deep learning model. In the solution, partitioned convolutions are executed based on an input and a set of parameter values of the convolutional layer sequentially in a trusted execution environment (TEE) of a computing device. The execution of a given one of partitioned convolutions comprises: storing, into a protected memory area in the TEE, an input portion of the input to be processed by a subset of parameter values for the given partitioned convolution; determining a result of the given partitioned convolution through a single matrix multiplication operation; and removing the input portion. By combining results of the partitioned convolutions, a result of the convolution is determined. Therefore, the solution can accelerate the model execution speed and improve the storage efficiency in a highly safe TEE with limited memory resources.