Compute-in-Memory Block Floating-Point Mantissa Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current compute-in-memory (CiM) architectures are limited to fixed-point computations, which restrict the precision range and accuracy of AI and machine learning models, especially in training processes, due to the lack of support for extended fixed-point and floating-point operations, leading to inefficiencies and accuracy losses.
Innovation Solution
The implementation of a CiM architecture that dynamically converts floating-point numbers to block floating-point format, allowing for extended fixed-point computations by partitioning mantissas into sub-words, using digital circuits to sequence and accumulate partial products, and employing redundancy and error correction schemes to prevent bit error amplification during mantissa renormalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed-point computations are used in CiM architectures, then energy efficiency and computational speed are improved, but precision range and accuracy are limited
Solution Approach 1:
The patent segments floating-point numbers into block floating-point format with separate exponent and mantissa components. The mantissa is further divided into sub-words that can be processed independently in parallel, enabling the CiM array to handle extended precision computations while maintaining high computational throughput through parallel processing of segmented data elements.
Solution Approach 2:
The patent introduces a new dimensional approach by adding exponent handling capabilities to the traditional fixed-point CiM architecture. Block floating-point format adds an exponent dimension to the fixed-point mantissa, allowing dynamic range extension without sacrificing the inherent speed and energy efficiency of fixed-point computations in the CiM array.
2Measurement precision
If floating-point operations are supported, then precision range and accuracy are improved, but device complexity and energy consumption increase
Solution Approach 1:
The patent implements dynamic exponent handling where the exponent value is determined based on the input data range and used to scale the block floating-point mantissas. This dynamic approach allows the same CiM hardware to adapt to different precision requirements without requiring multiple fixed configurations, reducing overall device complexity while maintaining high accuracy.
Solution Approach 2:
The patent introduces block floating-point format as an intermediary representation that bridges fixed-point and traditional floating-point operations. This intermediate format allows the CiM architecture to leverage the simplicity of fixed-point hardware while achieving floating-point precision through the block floating-point conversion process, avoiding the need for completely complex floating-point processing units.
3Measurement precision
If mantissa renormalization is performed, then computation accuracy is maintained, but bit error amplification risk increases
Solution Approach 1:
The patent performs preliminary error correction coding on the mantissa data before it enters the CiM array for computation and renormalization. By applying error correction codes in advance, the system can detect and correct bit errors that may occur during renormalization operations, maintaining computation accuracy while preventing error amplification without requiring complex real-time error monitoring during the critical renormalization process.
Data Source
AI summary
Systems, apparatuses and methods include technology that identifies workload numbers associated with a workload. The technology converts the workload numbers to block floating point numbers based on a division of mantissas of the workload numbers into sub-words and executes a compute-in memory operation based on the sub-words to generate partial products.