Block Floating Point Neural Network Exponent Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing floating-point representations in computer systems are inefficient in terms of memory usage and computational performance, especially in neural networks, due to the need for storing exponents with every floating-point number, which can lead to increased memory bandwidth requirements and computational overhead.
Innovation Solution
The implementation of Block Floating-Point (BFP) representations, where multiple elements share a common exponent, reducing memory usage and enabling simplified hardware implementation, energy reduction, and improved computational performance with minimal precision loss, by using a single exponent for groups of numbers with similar magnitudes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional floating-point representation is used with individual exponents for each number, then precision is maintained, but memory usage and computational overhead increase significantly
Solution Approach 1:
Multiple floating-point numbers are grouped into blocks that share a common exponent. Instead of storing separate exponents for each number, a single exponent represents all numbers in the block, significantly reducing memory usage while maintaining acceptable precision through the shared representation
Solution Approach 2:
The set of floating-point numbers is divided into multiple blocks, where each block contains numbers with similar magnitude characteristics. This segmentation allows each block to use its own common exponent, optimizing both memory efficiency and precision for different ranges of values
2Measurement precision
If traditional floating-point representation with individual exponents is used, then accuracy is preserved, but memory bandwidth requirements and computational overhead increase
Solution Approach 1:
By merging the exponent representation across multiple numbers within a block, the patent reduces the total number of exponent operations required during computation. This merging eliminates redundant exponent handling and reduces memory bandwidth requirements for transferring exponent data, thereby improving computational performance
3Measurement precision
If individual exponents are stored with every floating-point number, then precision is maintained, but hardware complexity and energy consumption increase
Solution Approach 1:
The patent merges exponent storage and processing by using a single common exponent for multiple floating-point numbers in a block. This reduces hardware complexity by eliminating the need for separate exponent storage units and processing logic for each number, while still maintaining precision through the shared exponent representation
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Apparatus and methods are disclosed for performing block floating-point (BFP) operations, including in implementations of neural networks. All or a portion of one or more matrices or vectors can share one or more common exponents. Techniques are disclosed for selecting the shared common exponents. In some examples of the disclosed technology, a method includes producing BFP representations of matrices or vectors, at least two elements of the respective matrices or vectors sharing a common exponent, performing a mathematical operation on two or more of the plurality of matrices or vectors, and producing an output matrix or vector. Based on the output matrix or vector, one or more updated common exponents are selected, and an updated matrix or vector is produced having some elements that share the updated common exponents.