Split Accumulator MAC Unit for Neural Network Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multiply accumulate (MAC) units face bandwidth constraints and power efficiency challenges due to the serial accumulation of products, which can lead to increased power consumption and reduced precision in neural network computations.
Innovation Solution
Implementing a split accumulator structure with two or more stages of accumulators, where one accumulator periodically transfers its running total to a second accumulator, decoupling memory bandwidth and allowing for a shared adder and multiplexor to switch accumulation operations between them, thereby reducing power consumption and accommodating a wider dynamic range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single accumulator is used in a MAC unit, then the structure is simple, but power consumption increases and precision is reduced due to continuous operation requirements
Solution Approach 1:
The accumulator is segmented into two distinct accumulators (first accumulator for partial sums, second accumulator for final accumulation) that operate in alternating phases. This segmentation allows the system to distribute the accumulation workload, enabling each accumulator to be in a low-power state during parts of the operation cycle, thereby reducing overall power consumption while maintaining computational functionality.
Solution Approach 2:
The system employs periodic action by alternating between two accumulation phases: when the first accumulator is actively accumulating partial sums, the second accumulator holds previously accumulated values, and vice versa. This periodic switching allows accumulators to enter idle or low-power states between active accumulation periods, reducing average power consumption compared to a single continuously operating accumulator.
2Device complexity
If a single accumulator is used in a MAC unit, then the structure is simple, but precision is reduced due to limited dynamic range handling
Solution Approach 1:
The accumulation process is segmented into two stages with two separate accumulators. The first accumulator handles partial sums with smaller dynamic range requirements, while the second accumulator handles the final accumulation with larger dynamic range. This segmentation allows each accumulator to be optimized for its specific range, improving overall precision without requiring a single high-capacity accumulator that would increase complexity and power consumption.
Solution Approach 2:
The system adds a temporal dimension to the accumulation process by using two accumulators operating in alternating time phases. This dimensional approach allows the system to handle wider dynamic ranges by distributing values across different time periods and accumulator instances, effectively increasing the system's computational precision capability without proportionally increasing the complexity of individual accumulator structures.
3Productivity
If memory bandwidth is coupled with accumulator operation, then data flow is efficient, but bandwidth constraints limit the operation of numerous MAC units
Solution Approach 1:
The accumulation function is segmented across two accumulators that can operate independently in alternating phases. This allows MAC units to continue producing results that are buffered in the first accumulator while the second accumulator processes previous data, effectively decoupling the continuous production capability from the periodic memory write requirement. This segmentation enables higher productivity without proportionally increasing memory bandwidth requirements.
Solution Approach 2:
The system performs preliminary accumulation in the first accumulator, building up partial sums before transferring them to the second accumulator for final processing and memory output. This preliminary action allows MAC units to maintain continuous operation and produce results faster than the memory can write, buffering intermediate results locally. This approach increases productivity by allowing computation to proceed without waiting for memory bandwidth availability.
Data Source
AI summary
In a multiply accumulate (MAC) unit, an accumulator may be implemented in two or more stages. For example, a first accumulator may accumulate products from the multiplier of the MAC unit, and a second accumulator may periodically accumulate the running total of the first accumulator. Each time the first accumulator's running total is accumulated by the second accumulator, the first accumulator may be initialized to begin a new accumulation period. In one embodiment, the number of values accumulated by the first accumulator within an accumulation period may be a user-adjustable parameter. In one embodiment, the bit width of the input of the second accumulator may be greater than the bit width of the output of the first accumulator. In another embodiment, an adder may be shared between the first and second accumulators, and a multiplexor may switch the accumulation operations between the first and second accumulators.