Neural Network Hardware Acceleration with Stochastic Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require significant computational resources and power due to the high complexity of multiply-and-accumulate (MAC) arithmetic units, which are major consumers of processing power and physical area in digital hardware accelerators, especially in applications like real-time object detection and segmentation in video streams.
Innovation Solution
A digital hardware circuit with a mixed array of multipliers and adders that routes input pairs to appropriate multiplier units based on estimated product magnitude, using small multipliers for smaller products and large multipliers for larger products, along with a sorting unit to efficiently process activation-weight pairs, reducing power consumption and chip area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fully accurate multiplication of larger values is supported, then computation accuracy is improved, but chip area and power consumption increase
Solution Approach 1:
The patent applies local quality by implementing different multiplier types (accurate and approximate) within the same array, allowing each multiplier to have properties tailored to its specific computational needs. Some multipliers provide full accuracy while others provide approximate computation, optimizing the trade-off between accuracy and area for different locations in the array.
Solution Approach 2:
The system dynamically selects between accurate and approximate multipliers based on the characteristics of the input values being processed. The selection is not static but adapts to the specific computational requirements of each operation, allowing the system to optimize between accuracy and area consumption in real-time.
2Measurement precision
If fully accurate multiplication of larger values is supported, then computation accuracy is improved, but power consumption increases
Solution Approach 1:
Different multipliers within the array have different power consumption characteristics tailored to their function. Approximate multipliers consume less power than accurate multipliers, and the system selectively activates the appropriate type based on computational needs, optimizing overall power usage.
Solution Approach 2:
The system dynamically adjusts power consumption by selecting between accurate and approximate multipliers based on the specific computational requirements of each operation, allowing power usage to adapt to the actual accuracy needs of the neural network computation.
3Productivity
If a large array of MAC units is used to support high computational load, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent introduces heterogeneity into the MAC unit array by implementing different types of multipliers (accurate and approximate) with different complexity characteristics. This allows the system to achieve high computational throughput while managing complexity through strategic placement of different unit types.
Solution Approach 2:
The system dynamically routes computational tasks to appropriate MAC units based on their capabilities and the requirements of the specific operation, allowing the system to optimize between productivity and complexity management through adaptive task allocation.
4Measurement precision
If high-precision floating-point circuits are used, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent implements a mixed-precision architecture where different parts of the computation use different precision levels. Not all multipliers use full floating-point precision; instead, approximate integer arithmetic is used where sufficient, reducing power consumption while maintaining adequate precision for the application.
Solution Approach 2:
The system changes the precision parameter dynamically, switching between full-precision floating-point and lower-precision integer representations based on the specific computational requirements. This allows the system to optimize power consumption by using lower precision when full precision is not necessary.
Data Source
AI summary
A digital circuit for accelerating computations of an artificial neural network model includes a pairs selection unit that selects different subsets of pairs of input vector values and corresponding weight vector values to be processed simultaneously at each time step; a sorting unit that simultaneously processes a vector of input-weight pairs wherein pair values whose estimated product is small are routed with a high probability to small multipliers, and pair values whose estimated product is greater are routed with a high probability to large multipliers that support larger input and output values; and a core unit that includes a plurality of multiplier units and a plurality of adder units that accumulate output results of the plurality of multiplier units into one or more output values that are stored back into the memory, where the plurality of multiplier units include the small multipliers and the large multipliers.