AI Rounding Circuits Using Re-Rounding to Cut Variance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional processors face inefficiencies in handling the complex algorithms and data-intensive workflows of artificial intelligence (AI) tasks due to their general-purpose architectures, failing to meet the demands of optimized performance and efficiency required by AI applications.
Innovation Solution
An AI-accelerating processor system with a systolic array and rounding circuits that perform matrix multiplication at reduced precision levels, utilizing multiple rounding circuits to generate different rounded values for computations, and processing elements to perform matrix operations using these values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional processors are used for AI computations, then general-purpose computing capability is maintained, but computational efficiency and processing speed for AI tasks deteriorate
Solution Approach 1:
The processor is divided into specialized functional units including a systolic array for parallel matrix operations, rounding circuits for precision conversion, and processing elements for specific computational tasks. This segmentation allows each unit to be optimized for its specific function while maintaining overall system versatility through coordinated operation of multiple specialized components.
2Measurement precision
If high precision computations are performed, then computational accuracy is improved, but processing speed and power efficiency deteriorate
Solution Approach 1:
The system dynamically changes the precision parameter of computations by incorporating multiple rounding circuits that can convert data between different precision levels. The processing elements perform computations at reduced precision levels (e.g., lower bit-depth) while maintaining sufficient accuracy for AI tasks, thereby achieving faster processing speeds and improved power efficiency without completely sacrificing computational accuracy.
3Measurement precision
If multiple rounding circuits generate different rounded values for the same input, then variance in computational results increases, but overall computational accuracy improves through error cancellation
Solution Approach 1:
The system converts the potentially harmful effect of variance introduced by multiple rounding circuits into a beneficial error cancellation mechanism. By having multiple rounding circuits generate different rounded values from the same input, the random variations in rounding errors tend to cancel each other out when results are aggregated, leading to improved overall computational accuracy compared to using a single deterministic rounding approach.
Data Source
AI summary
An AI-accelerating processor system may include memory that stores a value at a first precision level. The system may include a systolic array configured to perform computation. The systolic array may include rounding circuits. Each rounding circuit may round the value at the first precision level to a second precision level that is lower than the first precision level. At least a first rounding circuit and a second rounding circuit are configured to round the same value differently to respectively generate at least a first rounded value and a second rounded value. The systolic array may also include processing elements that are configured to receive a version of the value in one or more collective operations. At least a first processing element and a second processing element are configured to perform computations involving the value by respectively using the first rounded value and the second rounded value.