Sparse Matrix Compression Using Dictionary Tags for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning compression techniques face challenges in achieving full data throughput and optimized compression gains due to timing demands, data latency, and inefficient compression ratios, particularly in deep submicron silicon, where they often require two compression engines and introduce additional circuitry and power consumption, failing to effectively compress sparse data and meeting only 50% of speed targets.
Innovation Solution
A single compression engine utilizing a compressor dictionary with tags for zero, one, partial matches, and no matches is implemented, compressing data samples by storing tags and excluding identified values, providing optimized compression gains, lower latency, and reduced power consumption, applicable to both weights and activations in deep learning workloads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional deep learning compression techniques are used, then data can be compressed to save routing bandwidth and storage capacity, but timing demands are not met and data latency increases
Solution Approach 1:
The compression engine segments data processing into distinct phases: identifying frequently occurring values, assigning tags to these values, and replacing actual data with tags during compression. This segmentation allows the system to focus computational effort on the most compressible portions of the data while maintaining timing requirements.
Solution Approach 2:
The system performs preliminary analysis to identify frequently occurring values in the data stream before the actual compression operation. By pre-identifying these values and assigning them tags in advance, the compression engine can operate at full speed during the actual compression phase without timing violations.
2Speed
If two compression engines are used to meet timing demands, then speed targets can be achieved, but device complexity and power consumption increase
Solution Approach 1:
The compression engine is designed to be self-optimizing by automatically identifying frequently occurring values in the input data and adapting its compression strategy accordingly. This self-service capability allows a single engine to achieve the performance that previously required two engines, eliminating redundant hardware.
Solution Approach 2:
The system dynamically changes compression parameters based on the characteristics of the input data. By adjusting which values are tagged and how compression is applied based on data patterns, a single engine can optimize its throughput to meet timing demands without requiring multiple engines.
3Quantity of substance
If conventional compression techniques are used, then some compression is achieved, but compression gains are not optimized and sparse data is not effectively compressed
Solution Approach 1:
The compression engine applies different compression strategies to different portions of the data based on local characteristics. Frequently occurring values are identified and tagged with compact representations, while other data is handled differently. This local quality approach optimizes compression ratios by tailoring the compression method to the actual data distribution.
Solution Approach 2:
The system maintains continuous compression operation by processing data in a streamlined manner without interruption. The tag-based approach allows for continuous replacement of data values with compact tags, maintaining high compression throughput while achieving optimized compression ratios throughout the entire data stream.
Data Source
AI summary
An apparatus to facilitate deep learning numeric data and sparse matrix compression is disclosed. The apparatus includes a processor comprising a compression engine to: receive a data packet comprising a plurality of cycles of data samples, and for each cycle of the data samples: pass the data samples of the cycle to a compressor dictionary; identify, from the compressor dictionary, tags for each of the data samples, wherein the compressor dictionary comprises at least a first tag for data having a value of zero and a second tag for data having a value of one; and compress the data samples into compressed cycle data by storing the tags as compressed data, wherein the data samples identified with the first tag are compressed using the first tag and the data samples identified with the second tag are compressed using the second tag at the same time as values of the data samples identified with the first tag or the second tag are excluded from the compressed cycle data.