Hardware Dot-Product Acceleration Using Shared Exponents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication operations in machine learning applications are computationally resource-intensive, particularly for large matrices, requiring efficient computation to reduce memory and processor utilization.
Innovation Solution
A hardware accelerator is used to compute dot products by employing shared exponents for vector elements, allowing for faster matrix multiplication through reduced exponent normalization operations, and dynamically reconfiguring multiplier blocks to handle shared and unshared-exponent data types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional matrix multiplication is performed on large matrices, then computation accuracy is maintained, but computational resource utilization (memory and processor) becomes excessively high
Solution Approach 1:
The patent segments the matrix multiplication computation into multiple dot product operations, where each dot product is computed independently. This segmentation allows for optimized computation of each element while maintaining overall accuracy, reducing the burden on processors and memory by breaking down large matrix operations into manageable units.
Solution Approach 2:
The patent changes computational parameters by using shared exponents for vector elements and implementing dynamic reconfiguration of multiplier blocks. This parameter change optimizes the computation process by reducing redundant exponent normalization operations, thereby improving computational efficiency while maintaining precision through proper mantissa handling.
2Measurement precision
If exponent normalization operations are performed for each vector element, then computation precision is maintained, but computational overhead increases
Solution Approach 1:
The patent merges the exponent handling by using shared exponents for multiple vector elements. Instead of performing exponent normalization for each individual element, the system computes exponents once and reuses them across multiple elements, significantly reducing computational overhead while maintaining precision through separate mantissa computations.
Solution Approach 2:
The patent performs preliminary computation of shared exponents before the main dot product calculations. By pre-computing and storing exponent values that will be reused across multiple vector elements, the system eliminates redundant exponent normalization operations during the main computation phase, reducing overall computational overhead.
3Device complexity
If hardware accelerator uses fixed multiplier block configuration, then device complexity is reduced, but adaptability to different data types decreases
Solution Approach 1:
The patent implements dynamic reconfiguration of multiplier blocks within the hardware accelerator. The system can switch between different operational modes (shared-exponent mode and unshared-exponent mode) based on the input data type, allowing the same hardware to efficiently handle both data representations without requiring separate fixed configurations for each type.
Data Source
AI summary
A computing device, including a hardware accelerator configured to train a machine learning model by computing a first product matrix including a plurality of first dot products. Computing the first product matrix may include receiving a first matrix including a plurality of first vectors and a second matrix including a plurality of second vectors. Each first vector may include a first shared exponent and a plurality of first vector elements. Each second vector may include a second shared exponent and a plurality of second vector elements. For each first vector, computing the first product matrix may further include computing the first dot product of the first vector and a second vector. The first dot product may include a first dot product exponent, a first dot product sign, and a first dot product mantissa. Training the first machine learning model may further include storing the first product matrix in memory.