Sparse Tensor Storage Formats for Memory and Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for performing computations on large-scale sparse tensors are inefficient due to high memory requirements, leading to memory blowup issues and performance penalties, especially in applications like Tucker decomposition, where sparse matrix formats fail to account for tensor properties and do not support efficient storage of dense sub-tensors.
Innovation Solution
The introduction of new sparse tensor storage formats, such as the mode-generic and mode-specific formats, which reduce memory usage and improve data locality by storing frequently accessed data elements closer together, along with data reuse optimization techniques to minimize redundant computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sparse matrix formats are used to store sparse tensors, then storage efficiency is improved, but computation performance deteriorates due to poor data locality
Solution Approach 1:
The patent segments the sparse tensor storage into multiple formats (CSR, CSC, COO, DOK) and automatically selects the appropriate format based on the operation being performed. This segmentation allows each format to be optimized for specific access patterns, resolving the contradiction between storage efficiency and computation performance.
Solution Approach 2:
The patent implements dynamic format conversion, where the storage format is changed on-the-fly based on the computational requirements. For example, converting from CSR to COO format when random access is needed, or to CSR when sequential access is sufficient. This dynamic adaptation resolves the static contradiction between storage efficiency and computation performance.
2Productivity
If dense tensor storage is used, then computation performance is improved due to better data locality, but memory requirements increase significantly
Solution Approach 1:
The patent applies local quality by storing different portions of the tensor in different formats based on their access patterns. Frequently accessed dense sub-tensors are stored in dense format for performance, while sparse regions use sparse formats to save memory. This resolves the contradiction by applying different storage strategies to different parts of the data.
Solution Approach 2:
The patent changes the storage parameter (density) dynamically based on the computational needs. When a region requires frequent access, it is stored densely; when access is infrequent, it is stored sparsely. This parameter change resolves the contradiction between memory usage and computation performance.
3Adaptability or versatility
If format conversion is performed frequently, then adaptability to different operations is improved, but computation overhead increases
Solution Approach 1:
The patent performs preliminary actions by pre-converting tensors to a canonical format (such as COO) during data loading, and pre-computing format conversion indices. This preliminary preparation reduces the need for frequent conversions during computation, resolving the contradiction between adaptability and overhead.
Solution Approach 2:
The patent uses copying strategies where a tensor is converted to a different format only when necessary, and the converted version is cached for subsequent operations. This avoids repeated conversions and reduces overhead while maintaining adaptability to different operations.
Data Source
AI summary
In a system for storing in memory a tensor that includes at least three modes, elements of the tensor are stored in a mode-based order for improving locality of references when the elements are accessed during an operation on the tensor. To facilitate efficient data reuse in a tensor transform that includes several iterations, on a tensor that includes at least three modes, a system performs a first iteration that includes a first operation on the tensor to obtain a first intermediate result, and the first intermediate result includes a first intermediate-tensor. The first intermediate result is stored in memory, and a second iteration is performed in which a second operation on the first intermediate result accessed from the memory is performed, so as to avoid a third operation, that would be required if the first intermediate result were not accessed from the memory.