Transposable Sparse Tensor Storage With Indexed Non-Zero Packing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques for large machine learning models are inefficient in managing tensors with many zero-valued parameters, leading to high memory and bandwidth requirements, and existing methods do not effectively utilize sparsity in transposable matrices.
Innovation Solution
Implementing sparsity storage formats that encode datasets using metadata to describe indices of non-zero values and packed data elements, allowing for efficient storage and decoding of transposable matrices, reducing memory and bandwidth needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data compression techniques are used for machine learning models, then storage space is reduced, but memory and bandwidth requirements remain high due to inefficient sparsity management
Solution Approach 1:
The patent segments the tensor data into non-zero elements and zero elements, storing only non-zero elements in compressed format along with metadata indicating their positions. This segmentation allows the system to reduce storage space for sparse data while maintaining efficient access patterns that lower memory and bandwidth requirements during operations.
Solution Approach 2:
The patent applies different storage strategies to different parts of the data structure: non-zero elements are stored in a compressed format with metadata, while zero elements are implicitly represented. This local quality approach optimizes storage and access for the specific characteristics of sparse tensor data, reducing overall memory and bandwidth requirements.
2Reliability
If sparsity is not utilized in transposable matrices, then data accuracy is maintained, but memory size and operational bandwidth increase
Solution Approach 1:
The patent extracts and stores only the non-zero elements of the tensor along with metadata indicating their positions. By taking out only the essential non-zero data and its positional information, the system maintains complete data accuracy for computations while significantly reducing memory size compared to storing all elements including zeros.
3Loss of information
If all data values are stored, then complete data is available, but storage efficiency decreases when data contains many zero-valued parameters
Solution Approach 1:
The patent creates a compressed representation of the tensor that copies only the non-zero elements along with metadata about their positions. This copying strategy preserves all necessary information for accurate reconstruction and computation while improving storage efficiency by eliminating redundant zero-valued parameters from the stored data.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
Techniques for manipulating sparsity storage architectures are provided. One aspect includes a computing device for implementing sparsity storage architectures, the computing device comprising processing circuitry and memory comprising instructions that, during execution, causes the processing circuitry to receive a dataset comprising data values; encode the received dataset by identifying a subset of the data values in the received dataset; and generating metadata describing indices of the subset of the data values in the received dataset; and store the encoded dataset comprising the metadata and packed data, wherein the packed data corresponds to the subset of the data values, and wherein the encoded dataset is capable of being decoded into a transposable data format with a same sparsity level in transposed and non-transposed forms.