Dual-Path MAD Circuit for Faster Neural Network FP Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks, particularly large language models (LLMs), require significant computing power and energy consumption due to extensive floating-point operations, which are computationally intensive and inefficient.
Innovation Solution
Implementing a multiply-add (MAD) circuit with two data paths in a multiply-accumulate (MAC) circuit to perform floating-point operations, allowing concurrent processing of input data with different kernel coefficients, shifting, and adding to align with fixed-point precision, thereby reducing processing cycles, bandwidth, and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point operations are used for neural network computations, then precision is improved, but computing power and energy consumption increase significantly
Solution Approach 1:
The patent changes the numerical parameter format from standard floating-point to a customized floating-point representation with adjusted exponent and mantissa bit allocations. This parameter change enables more efficient hardware implementation that reduces energy consumption while preserving computational precision for neural network operations
Solution Approach 2:
The patent replaces complex floating-point arithmetic operations with simplified hardware circuits including shift registers and adders. By substituting the traditional floating-point unit mechanics with these simpler components, the system achieves comparable precision with significantly reduced energy consumption
2Measurement precision
If floating-point operations are used for neural network computations, then precision is improved, but processing speed decreases due to computational intensity
Solution Approach 1:
The patent replaces complex floating-point multiplication and division operations with simpler shift and add operations. This mechanical substitution in the computational process dramatically increases processing speed while maintaining sufficient precision for neural network computations through proper fixed-point representation
Solution Approach 2:
The patent changes the computational parameters by using customized floating-point formats with optimized exponent and mantissa configurations. This parameter optimization enables faster convergence and fewer processing cycles while preserving the necessary precision for accurate neural network inference
3Measurement precision
If standard floating-point format is used, then precision is maintained, but bandwidth and memory requirements increase
Solution Approach 1:
The patent changes the data format parameters by implementing a customized floating-point representation with optimized bit allocation. This parameter change reduces the quantity of data that needs to be stored and transmitted while maintaining the precision necessary for neural network operations through efficient use of each bit
4Device complexity
If conventional MAC circuit is used, then simplicity is maintained, but throughput for floating-point operations is limited
Solution Approach 1:
The patent segments the computation into multiple parallel data paths within the MAC circuit, each handling different portions of the neural network computation. This segmentation enables concurrent processing of multiple operations, doubling the throughput while keeping each individual path relatively simple
Solution Approach 2:
The patent adds a parallelism dimension to the MAC circuit by implementing multiple data paths that operate simultaneously. This dimensional expansion from sequential to parallel processing increases throughput without significantly increasing the complexity of individual circuit components
Data Source
AI summary
Disclosed herein are systems and methods for performing floating point (FP) operations in a NN. For example, a MAD circuit includes two data paths. Each of the data paths is configured to receive the same input data and multiply the same input data by different kernel coefficients to generate respective FP values. Each data path shifts its respective FP value to generate a respective shifted value that is aligned with a fixed point precision of an accumulator. One data path obtains data from a first set of register files of the accumulator, aggregates the data with its shifted value, and stores the resulting value in the first set of register files. The other data path obtains data from a second set of register files, aggregates the data with its shifted value, and stores the resulting value in the second set of register files.