Neural Processing Circuit for Shared Outlier Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computations are inefficient due to the processing of activation values that exceed a threshold, leading to unnecessary resource utilization in multipliers.
Innovation Solution
A method and system that separates activations into least and most significant parts, using dedicated multipliers for the least significant parts and a shared multiplier for the most significant parts, with a buffer to store excess activations for efficient handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a 5×8 bit multiplier is used to multiply all activations, then all activations can be processed uniformly, but resources are wasted when activations are actually 4 bits wide
Solution Approach 1:
The activation values are segmented into two groups: outliers (activations exceeding the threshold) and non-outliers (activations within the threshold). This segmentation allows the system to apply different processing methods to different groups, using smaller multipliers for non-outliers and larger multipliers for outliers, thereby avoiding resource wastage while maintaining correctness.
Solution Approach 2:
Different multiplier configurations are applied to different activation values based on their local characteristics. Non-outlier activations use 4×8 bit multipliers, while outlier activations use 5×8 bit multipliers. This local adaptation of processing resources matches the actual data distribution, eliminating the need for uniform over-provisioning.
2Productivity
If separate multipliers are used for each row to handle outliers, then computational efficiency is maintained, but device complexity increases
Solution Approach 1:
Multiple row-specific multipliers are merged into a single shared multiplier resource. The shared multiplier is dynamically allocated to handle outlier activations from different rows as they are detected, eliminating the need for dedicated multipliers in each row while maintaining computational efficiency. This sharing strategy reduces device complexity by consolidating redundant hardware resources.
Solution Approach 2:
The multiplier configuration is made dynamic rather than static. The system continuously monitors activation values and dynamically adjusts the multiplier bit-width based on whether the current activation is an outlier or not. This dynamic adaptation allows the system to maintain high computational efficiency for non-outliers while handling outliers correctly, without requiring a fixed complex configuration for all cases.
Data Source
AI summary
A system and method for handling processing with outliers. In some embodiments, the method includes: reading a first activation and a second activation, each including a least significant part and a most significant part, multiplying a first weight and a second weight by the respective activations, the multiplying of the first weight by the first activation including multiplying the first weight by the least significant part of the first activation in a first multiplier, the multiplying of the second weight by the second activation including: multiplying the second weight by the least significant part of the second activation in a second multiplier, and multiplying the second weight by the most significant part of the second activation in a shared multiplier, the shared multiplier being associated with a plurality of rows of an array of activations.