Block Floating Point Vector Decomposition for Neural Network Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks using block floating point format face accuracy issues due to lowered precision, which affects system performance and accuracy in tasks like object recognition and speech recognition, despite increased processing speed.
Innovation Solution
The method involves decomposing block floating point vectors into smaller bit-width vectors, performing dot product operations, and reconstructing higher bit-width vectors to maintain accuracy while using lower-precision hardware, allowing for higher precision arithmetic with reduced computational burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If block floating point format is used with lowered precision, then processing speed is increased, but system accuracy is worsened
Solution Approach 1:
The patent divides the block floating point vector into multiple sub-vectors with reduced bit-width mantissas. Each sub-vector processes a portion of the original vector, allowing parallel computation that increases processing speed while maintaining accuracy through the accumulation of results from multiple precision-reduced operations.
Solution Approach 2:
The patent changes the bit-width parameter of the mantissa by decomposing the original block floating point vector into sub-vectors with smaller mantissa bit-widths. This parameter change enables faster processing on lower-precision hardware while the block structure allows accuracy to be preserved through the collective processing power of multiple sub-vectors.
2Productivity
If block floating point vectors are decomposed into reduced bit-width vectors, then computational burden is reduced, but the complexity of the computation process is worsened
Solution Approach 1:
The computation process is segmented into multiple independent dot product operations on sub-vectors, each with reduced bit-width mantissas. This segmentation allows the complex computation to be divided into simpler, parallelizable operations that can be executed more efficiently on lower-precision hardware.
Solution Approach 2:
The block floating point vector is decomposed into sub-vectors with reduced bit-width mantissas before performing the dot product operations. This preliminary decomposition simplifies the subsequent computations by reducing the bit-width requirements, enabling the use of less powerful and more efficient hardware.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A system for block floating point computation in a neural network receives a block floating point number comprising a mantissa portion. A bit-width of the block floating point number is reduced by decomposing the block floating point number into a plurality of numbers each having a mantissa portion with a bit-width that is smaller than a bit-width of the mantissa portion of the block floating point number. One or more dot product operations are performed separately on each of the plurality of numbers to obtain individual results, which are summed to generate a final dot product value. The final dot product value is used to implement the neural network. The reduced bit width computations allow higher precision mathematical operations to be performed on lower-precision processors with improved accuracy.