DNN Accelerator Fixed-Point Emulation of Floating-Point Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face significant energy costs and inefficiencies due to high computing demands, particularly in matrix multiplication operations, which current hardware accelerators struggle to optimize effectively, leading to hardware complexity, inefficiency in runtime power, and limited performance.
Innovation Solution
An DNN accelerator that enables fixed-point emulation of floating-point computation by extracting extreme exponents from floating-point matrices and converting them into fixed-point vectors for processing, allowing for efficient matrix multiplication within a systolic array designed for integer tensor computation without significant hardware changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point computation is implemented directly in systolic array hardware, then computation accuracy is improved, but hardware complexity and power consumption increase significantly
Solution Approach 1:
The patent introduces an intermediary conversion process that transforms floating-point numbers into a specialized format (e.g., block floating-point or fixed-point with exponent tags) before processing in the systolic array. This intermediary representation allows the array to handle floating-point computations using simpler fixed-point arithmetic logic, thereby maintaining accuracy while reducing hardware complexity. The conversion layers act as mediators that bridge the gap between floating-point precision requirements and fixed-point hardware simplicity.
2Adaptability or versatility
If floating-point support is added to systolic array hardware, then computation versatility is improved, but runtime power consumption increases
Solution Approach 1:
The patent dynamically changes the numerical representation parameters by converting floating-point inputs into an alternative format (such as block floating-point where groups of numbers share a common exponent) before processing. This parameter transformation allows the systolic array to maintain versatility in handling different floating-point ranges and magnitudes while operating in a lower-power fixed-point or integer arithmetic mode during the actual computation phases, thereby reducing runtime power consumption.
3Productivity
If floating-point computation is performed in systolic arrays, then computational capability is improved, but hardware area increases
Solution Approach 1:
The patent employs copying by creating intermediate representations of floating-point data in a compact format (such as block floating-point with shared exponents) that can be processed by the existing fixed-point systolic array infrastructure. This copying approach allows the system to maintain full floating-point computational capability while utilizing the same physical hardware resources, effectively avoiding the need to duplicate hardware components and thereby preventing hardware area expansion.
4Measurement precision
If high-precision floating-point operations are performed, then deep learning accuracy is improved, but energy cost increases significantly
Solution Approach 1:
The patent dynamically adjusts numerical precision parameters by converting high-precision floating-point operations into lower-precision fixed-point or block floating-point computations during inference. By changing the precision parameters adaptively based on the computational requirements of different layers and operations, the system maintains sufficient deep learning accuracy while dramatically reducing the energy cost associated with high-precision floating-point arithmetic in the energy-constrained inference phase.
Data Source
AI summary
An DNN accelerator can perform fixed-point emulation of floating-point computation. In a multiplication operation on two floating-point matrices, the DNN accelerator determines an extreme exponent for a row in the first floating-point matrix and determines another extreme exponent for a column in the second floating-point matrix. The row and column can be converted to fixed-point vectors based on the extreme exponents. The two fixed-point vectors are fed into a PE array in the DNN accelerator. The PE array performs a multiplication operation on the two fixed-point vectors and generates a fixed-point inner product. The fixed-point inner product can be converted back to a floating-point inner product based on the extreme exponents. The floating-point inner product is an element in the matrix resulted from the multiplication operation on the two floating-point matrices. The matrix can be accumulated with another matrix resulted from a fixed-point emulation of a floating-point matrix multiplication.