Floating-Point Compression Using Exponent-Based Bit Downscaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high bandwidth usage and power consumption associated with processing floating-point numbers in computing systems, particularly in GPUs and NPUs, due to limited bus bandwidth, which leads to latency issues during data storage and retrieval.
Innovation Solution
A method that involves pre-processing floating-point numbers to reduce their dynamic range, setting fractional components to 0 when near 0 or 1, and downscaling the fractional bit sequence based on exponential component thresholds, followed by encoding and transmitting the data to reduce the number of bits required, thereby reducing bus bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point numbers are processed using standard formats (FP16, FP32), then computational accuracy is maintained, but bus bandwidth usage increases and power consumption rises
Solution Approach 1:
The patent changes the parameter of floating-point representation by introducing a compressed format that modifies the bit allocation structure. The exponential component uses fewer bits (e.g., 4 bits instead of 5 or 8) and the fractional component uses variable-length encoding based on the exponential value, thereby reducing the total bit width while maintaining sufficient precision for computational tasks.
Solution Approach 2:
The patent segments the floating-point number into distinct components (exponential component and fractional component) and applies different encoding strategies to each. The exponential component is encoded with fixed fewer bits, while the fractional component uses variable-length encoding where less significant bits are omitted when the exponential value indicates smaller magnitude, thereby reducing overall precision requirements for certain ranges.
2Productivity
If floating-point numbers are stored and retrieved from memory, then data processing can proceed, but bus bandwidth is consumed and latency increases
Solution Approach 1:
The patent changes the data size parameter by compressing floating-point numbers into fewer bits. This reduction in bit width directly decreases the amount of data that must be transferred over the bus between memory and processor, thereby reducing both bandwidth consumption and transfer time, which lowers latency and improves overall data processing throughput.
3Measurement precision
If full precision floating-point representation is used, then data quality is maintained, but the number of bits transmitted over the bus increases
Solution Approach 1:
The patent changes the representation parameters by using a variable precision approach where the number of bits allocated to the fractional component depends on the exponential component's value. When the exponential indicates a smaller magnitude, fewer fractional bits are used, reducing the total bit count while maintaining adequate precision for the given range of values.
Solution Approach 2:
The patent applies partial precision encoding by selectively omitting less significant fractional bits when the exponential component indicates that such precision is not necessary for the current value range. This partial action approach transmits only the necessary number of bits to maintain data quality for each specific case, rather than always transmitting full precision.
Data Source
AI summary
Certain aspects of the present disclosure provide a method of encoding data. The method generally includes receiving data comprising a fractional number comprising an exponential component and a fractional component, the exponential component being represented by an exponential bit sequence, the fractional component being represented by a fractional bit sequence. The method further includes determining if the fractional component is within a threshold of 0 or 1. The method further includes setting the fractional component to 0 when the fractional component is within the threshold of 0 or 1. The method further includes downscaling the fractional bit sequence based on a difference between the exponential component and a second threshold. The method further includes encoding the data. The method further includes transmitting the encoded data.