Sparse Tensor Storage for Transposable Data Layouts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques for large machine learning models are inefficient in managing tensors with many zero-valued parameters, leading to high memory and bandwidth requirements.
Innovation Solution
Implementing sparsity storage formats that utilize metadata to indicate zero-valued parameters, allowing for efficient encoding and decoding of sparse tensors, reducing memory size and bandwidth needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing data compression techniques are used for large machine learning models, then storage efficiency is improved, but efficiency in managing tensors with many zero-valued parameters deteriorates
Solution Approach 1:
The patent segments the tensor data into two distinct components: sparse data (non-zero values) and dense data (zero values). This segmentation allows each component to be stored and processed separately, with the sparse data stored in compressed format and zero-valued parameters tracked through metadata, thereby improving both storage efficiency and operational efficiency for sparse tensors
Solution Approach 2:
The patent extracts and separates the zero-valued parameters from the main tensor data. By identifying and extracting these zero values, the system can store only the non-zero values in compressed format while maintaining the ability to reconstruct the original tensor structure through metadata that tracks the positions of zero-valued parameters
2Reliability
If all data values are stored in full format, then data integrity is maintained, but memory size and bandwidth requirements increase
Solution Approach 1:
The patent extracts only the essential information needed to represent the data - specifically, the non-zero values and their positions. By extracting and storing only this critical information in compressed format, the system maintains data integrity while significantly reducing memory size and bandwidth requirements compared to storing all data values in full format
Solution Approach 2:
The patent changes the representation parameters of the data by transitioning from storing complete tensor values to storing compressed representations that include only non-zero values and metadata about zero-valued parameter positions. This parameter change enables more efficient use of memory and bandwidth while preserving the ability to reconstruct the original data
Data Source
AI summary
Techniques for manipulating sparsity storage architectures are provided. One aspect includes a computing device for implementing sparsity storage architectures, the computing device comprising processing circuitry and memory comprising instructions that, during execution, causes the processing circuitry to receive a dataset comprising data values; encode the received dataset by identifying a subset of the data values in the received dataset; and generating metadata describing indices of the subset of the data values in the received dataset; and store the encoded dataset comprising the metadata and packed data, wherein the packed data corresponds to the subset of the data values, and wherein the encoded dataset is capable of being decoded into a transposable data format with a same sparsity level in transposed and non-transposed forms.