Tile Tensor Interleaving for Efficient FHE Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fully homomorphic encryption (FHE) schemes are computationally expensive for large neural networks, limiting their use in secure data processing and storage due to high computational overhead, prompting the need for efficient data packing techniques to optimize operations like convolution.
Innovation Solution
The implementation of tile tensors with interleaved tiling and a packing optimizer that maps input tensors into non-contiguous slots within tile tensors, allowing for efficient high-level tensor manipulation and convolution operations over encrypted data, reducing computational overhead and optimizing for latency, throughput, and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If fully homomorphic encryption is used for neural network computations, then data privacy is preserved, but computational overhead increases significantly
Solution Approach 1:
The patent segments tensors into tile tensors with specific shapes and interleaving strides, organizing data into structured blocks that optimize FHE operation efficiency. This segmentation allows for better memory access patterns and reduces the computational overhead of homomorphic operations while maintaining data privacy.
Solution Approach 2:
The patent introduces tile tensor shapes and interleaving strides as additional dimensional parameters beyond the standard tensor dimensions. This dimensional extension enables more flexible data organization and optimization of FHE computations, allowing efficient packing and unpacking operations that reduce computational overhead.
2Ease of manufacture
If traditional tensor packing is used in FHE systems, then implementation is simple, but convolution operations require excessive rotations and multiplications
Solution Approach 1:
The patent performs preliminary organization of tensors into tile tensors with predetermined shapes and interleaving strides before convolution operations. This pre-structuring of data ensures that subsequent convolution operations can be executed efficiently with minimal rotations and multiplications, while maintaining implementation feasibility through systematic packing algorithms.
Solution Approach 2:
The patent changes the packing parameters by introducing tile tensor shapes and interleaving strides, transforming the traditional packing approach into an optimized structure. This parameter modification enables efficient convolution operations by aligning data layouts with the computational requirements of FHE-based neural networks.
3Productivity
If tile tensors with interleaved tiling are implemented, then convolution operations become efficient, but data structure complexity increases
Solution Approach 1:
The patent applies local quality by using different tile tensor shapes and interleaving strides for different dimensions and regions of the tensor. This localized optimization allows efficient convolution operations in specific areas while maintaining overall structure manageability, balancing convolution efficiency with data structure complexity.
Solution Approach 2:
The patent introduces dynamic parameters including tile tensor shapes and interleaving strides that can be adjusted based on computational requirements. This dynamic structure allows the system to adapt to different convolution scenarios, optimizing efficiency while managing complexity through configurable parameters rather than fixed rigid structures.
Data Source
AI summary
A method comprising: receiving an input tensor having a shape defined by [n1, . . . , nk], where k is equal to a number of dimensions that characterize the input tensor; receiving tile tensor metadata comprising: a tile tensor shape defined by [t1, . . . , tk], and information indicative of an interleaving stride to be applied with respect to each dimension of the tile tensor; constructing an output tensor comprising a plurality of the tile tensors, by applying a packing algorithm which maps each element of the input tensor to at least one slot location of one of the plurality of tile tensors, based on the tile tensor shape and the interleaving stride, wherein the interleaving stride results in non-contiguous mapping of the elements of the input tensor, such that each of the tile tensors includes a subset of the elements of the input tensor which are spaced within the input tensor according to the interleaving stride.