Neural Network Data Compression Using Block Floating Point Exponents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods for neural networks result in large numerical representations that exceed storage capacity, incur high computational costs, and cause bandwidth limitations, making them inefficient for storage and transfer.
Innovation Solution
The implementation of block floating point (BFP) encoding, which compresses data by finding common exponents within blocks and quads, storing only the differences or indicators of these exponents, thereby reducing the number of bits required for storage and enabling efficient matrix multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data compression methods are used for neural networks, then storage capacity is reduced, but the numerical representations require a large number of bits that still exceed storage capacity
Solution Approach 1:
The patent changes the numerical representation format from standard floating-point to block floating-point format, where a block of values shares a common exponent. This parameter change in representation method reduces the bits required per value while maintaining numerical precision through the shared exponent mechanism.
Solution Approach 2:
The patent segments the data into blocks where each block shares a common exponent value. By dividing the data structure into manageable blocks and representing each block with a shared exponent, the overall storage requirements are reduced while maintaining the ability to represent a wide dynamic range of values.
2Quantity of substance
If compressed numerical representations with large number of bits are used, then storage capacity is overwhelmed, but computational costs for training neural networks increase
Solution Approach 1:
The patent changes the numerical format to block floating-point with compressed exponents, reducing the bit-width from standard 32-bit or 64-bit floating-point to a compressed format. This parameter change simultaneously reduces storage requirements and computational energy by decreasing the number of bits that need to be processed during training operations.
3Quantity of substance
If numerical representations requiring large number of bits are used, then storage efficiency is reduced, but data transfer bandwidth is limited
Solution Approach 1:
The patent changes the data representation to a compressed block floating-point format that requires fewer bits per value. This parameter change directly improves data transfer speed by reducing the volume of data that must be transmitted across bandwidth-limited connections, while maintaining storage efficiency through the compressed representation.
4Reliability
If standard floating-point representation is used, then numerical stability is maintained, but the number of bits required is large
Solution Approach 1:
The patent changes from standard floating-point to block floating-point representation, maintaining numerical stability through the shared exponent mechanism that preserves the dynamic range and precision characteristics of standard floating-point, while reducing the total number of bits required by compressing the exponent storage at the block level.
Data Source
AI summary
Systems and methods for generating a representative value of a data set by first compressing a portion of values in the data set to determine a first common value and further compressing a subset of the portion of values to determine a second common value. The representative value is generated by taking the difference between the first common value and the second common value, wherein the representative value corresponds to a mathematical relationship between the first and second common values and each value within the subset of the portion of values. The representative value requires less storage than the first and second common values.