Activation Value Compression Using Zero and Exponent Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning operations require significant computing resources due to the large amount of data involved, leading to strain on computing resources, and existing compression methods do not effectively address this issue.
Innovation Solution
A data compression technique that identifies consecutive identical non-zero values, exponent values, and zero values to represent them using control values and data items, achieving lossless compression by reducing the size of activation values in machine learning workloads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If data compression is applied to machine learning activation values, then bandwidth utilization and power consumption are reduced, but data integrity may be compromised
Solution Approach 1:
The patent segments the activation data into blocks and applies different compression techniques to different portions within each block. Specifically, it identifies and compresses sequences of identical values, zeros, and exponent values separately using targeted encoding schemes, thereby achieving effective compression while preserving data integrity through lossless methods
Solution Approach 2:
The patent changes the representation parameters of the data by transforming activation values into compressed formats that encode only essential information. It modifies how data is stored and transmitted by using control values to indicate patterns and properties rather than storing complete numerical values, achieving compression ratio greater than one while maintaining full data fidelity
2Quantity of substance
If data compression is applied to machine learning activation values, then bandwidth utilization is reduced, but compression complexity increases
Solution Approach 1:
The patent divides the compression process into distinct segments that handle different data patterns separately: identifying sequences of identical values, detecting zero values, and processing exponent values. This segmentation allows the system to apply appropriate compression techniques to each pattern type, reducing overall complexity compared to applying a single complex algorithm to all data
Solution Approach 2:
The patent extracts and removes redundant information from the activation data by identifying and compressing repeated patterns, sequences of zeros, and common exponent values. It takes out only the essential information needed to reconstruct the original data, achieving significant reduction in data volume while maintaining manageable compression complexity through targeted extraction of redundancies
3Quantity of substance
If existing compression methods are used, then some data reduction is achieved, but compression ratio is insufficient to significantly reduce computing resource strain
Solution Approach 1:
The patent applies different compression qualities and techniques to different local portions of the data based on their characteristics. It identifies specific patterns such as sequences of identical values, blocks of zeros, and common exponent values, and applies tailored compression schemes to each pattern type, achieving higher overall compression ratios than uniform compression methods
Solution Approach 2:
The patent merges multiple compression techniques into a unified approach that handles different data patterns simultaneously. It combines run-length encoding for identical sequences, sparse encoding for zero values, and exponent compression for floating-point values, creating a hybrid compression system that achieves significantly higher compression ratios than individual methods alone, thereby substantially reducing computing resource strain
Data Source
AI summary
Techniques are disclosed for compressing data. The techniques include identifying, in data to be compressed, a first set of values, wherein the first set of values include a first number of two or more consecutive identical non-zero values; including, in compressed data, a first control value indicating the first number of non-zero values and a first data item corresponding to the consecutive identical non-zero values; identifying, in the data to be compressed, a second value having an exponent value included in a defined set of exponent values; including, in the compressed data, a second control value indicating the exponent value and a second data item corresponding to a portion of the second value other than the exponent value; and including, in the compressed data, a third control value indicating a third set of one or more consecutive zero values in the data to be compressed.