Block Floating Point Compression for Neural Network Exponents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in compressing and decompressing data for neural networks, leading to excessive storage requirements and high computational costs due to the large number of bits needed to represent numerical data.
Innovation Solution
The implementation of a block floating point (BFP) encoder that generates compressed numerical representations by finding common exponent values within blocks and quads of a data set, allowing for efficient storage and computation by storing indicators of mathematical relationships rather than full exponent values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full precision numerical representations are used to represent data elements in neural networks, then measurement precision is improved, but storage capacity requirements increase and computational costs increase
Solution Approach 1:
The patent segments the exponent representation into multiple levels: a first exponent value applied to a block of data elements, and second exponent values applied to subsets within blocks. This hierarchical segmentation allows different precision levels for different data regions, reducing overall storage requirements while maintaining necessary precision where needed.
Solution Approach 2:
The patent applies local quality by allowing different exponent precision levels for different regions of the data set. Each block or subset can have its own exponent values, enabling high precision locally where required while using lower precision elsewhere, thus optimizing the trade-off between precision and storage capacity.
2Measurement precision
If full precision numerical representations are used to represent data elements in neural networks, then measurement precision is improved, but computational costs increase
Solution Approach 1:
The computational processing is segmented into multiple passes: a first pass processes data using the first exponent value at lower computational cost, while a second pass processes subsets using second exponent values for higher precision when needed. This segmentation reduces overall computational costs while maintaining precision requirements.
Solution Approach 2:
The patent applies partial action by using lower precision exponent values for the majority of data elements in blocks, and only applying higher precision second exponent values to specific subsets where needed. This avoids the excessive computational cost of applying full precision universally while still meeting precision requirements where necessary.
3Quantity of substance
If compressed numerical representations with fewer bits are used, then storage capacity requirements are reduced, but loss of information may occur
Solution Approach 1:
The patent segments the data representation into multiple exponent levels, using a first exponent value for general compression and second exponent values for specific subsets. This segmentation enables lossless or minimal-loss compression by preserving full precision information where needed while achieving compression elsewhere.
Solution Approach 2:
The patent creates a composite representation system combining first exponent values and second exponent values in a unified data structure. This composite approach allows the system to leverage the compression benefits of fewer bits while maintaining information integrity through the hierarchical exponent structure that preserves necessary precision.
4Quantity of substance
If compressed numerical representations with fewer bits are used, then storage capacity requirements are reduced, but bandwidth limitations become more significant
Solution Approach 1:
The patent segments data transfer into multiple passes, with a first pass transferring compressed data using first exponent values, and a second pass transferring additional precision information for subsets using second exponent values. This segmentation optimizes bandwidth utilization by transferring the majority of data at lower precision while selectively transferring additional precision information where needed.
Data Source
AI summary
Systems and methods for generating a representative value of a data set by first compressing a portion of values in the data set to determine a first common value and further compressing a subset of the portion of values to determine a second common value. The representative value is generated by taking the difference between the first common value and the second common value, wherein the representative value corresponds to a mathematical relationship between the first and second common values and each value within the subset of the portion of values. The representative value requires less storage than the first and second common values.