Split Accumulator MAC Architecture for Lower Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multiply accumulate (MAC) units face bandwidth constraints and power efficiency challenges due to memory requirements for accumulating outputs from numerous units, particularly in neural network computations.
Innovation Solution
Implementing a MAC unit with two or more stages of accumulators, where a first accumulator accumulates products and a second accumulator periodically accumulates the running total, decoupling memory bandwidth and using a shared adder and multiplexor to switch accumulation operations between stages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single accumulator is used in each MAC unit, then the device complexity is low, but the memory bandwidth requirement increases significantly when numerous MAC units need to read outputs simultaneously
Solution Approach 1:
The accumulator is divided into multiple stages (first accumulator and second accumulator) that operate in sequence. The first accumulator performs initial accumulation and then hands off to the second accumulator, which completes the accumulation process. This segmentation allows the memory to be accessed less frequently, reducing bandwidth requirements while maintaining functional simplicity.
2Device complexity
If a single accumulator is used in each MAC unit, then the device complexity is low, but the power consumption increases due to continuous memory operations
Solution Approach 1:
By segmenting the accumulator into two stages, the system reduces the frequency of memory access operations. The first accumulator handles initial computations locally, then transfers to the second accumulator which completes the task. This reduces power consumption by minimizing memory transactions while keeping the overall structure relatively simple.
Solution Approach 2:
The second accumulator acts as an intermediary between the first accumulator and the final memory output. It receives partial results from the first accumulator and completes the accumulation process, reducing the need for continuous memory access and thereby reducing power consumption.
3Productivity
If memory bandwidth is increased to accommodate numerous MAC units, then the productivity improves, but the device complexity and resource utilization increase
Solution Approach 1:
The accumulator is segmented into two stages that can operate in a pipelined manner. While the first accumulator is processing one set of data, the second accumulator is preparing to receive and complete the accumulation. This segmentation enables higher productivity without requiring increased memory bandwidth, as the pipelined structure allows overlapping operations.
Data Source
AI summary
In a multiply accumulate (MAC) unit, an accumulator may be implemented in two or more stages. For example, a first accumulator may accumulate products from the multiplier of the MAC unit, and a second accumulator may periodically accumulate the running total of the first accumulator. Each time the first accumulator's running total is accumulated by the second accumulator, the first accumulator may be initialized to begin a new accumulation period. In one embodiment, the number of values accumulated by the first accumulator within an accumulation period may be a user-adjustable parameter. In one embodiment, the bit width of the input of the second accumulator may be greater than the bit width of the output of the first accumulator. In another embodiment, an adder may be shared between the first and second accumulators, and a multiplexor may switch the accumulation operations between the first and second accumulators.