In-Memory MAC Architecture With Local Partial Sum Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning processing systems face challenges in efficiently handling partial sums in weight-stationary schemes, leading to energy and latency penalties due to the offloading and reloading of partial accumulation results, which can reduce the area advantage and introduce throughput bottlenecks.
Innovation Solution
A neural-network-processing architecture that includes a first accumulator circuit, flip-flop array, write register, and multiplexer, along with a systolic flow architecture that allows concurrent multiply-and-accumulate operations and manages partial sums without throughput penalties by integrating these operations directly into the processing system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If partial sums are offloaded to memory and reloaded for accumulation, then the processing can be distributed, but energy consumption increases and latency is introduced
Solution Approach 1:
The patent merges the partial sum accumulation function directly into the processing elements (PEs) by providing each PE with local accumulator circuits. This eliminates the need to offload partial sums to external memory and reload them, as the accumulation occurs in-place within the processing element itself, thereby reducing both energy consumption and latency associated with memory access operations.
Solution Approach 2:
The patent introduces a new dimension to the processing architecture by adding temporal multiplexing capabilities to the accumulators. The accumulators can operate in different modes (accumulation mode, loading mode, clearing mode) within the same time frame, allowing the system to handle multiple operations efficiently without requiring separate physical structures for each function, thus reducing overall system complexity while maintaining high performance.
2Productivity
If dedicated hardware accelerators are used for machine learning processing, then processing capacity increases, but space and power requirements increase
Solution Approach 1:
The patent designs processing elements that can perform multiple functions: they can execute neural network inference operations, perform partial sum accumulation, and manage data flow through integrated multiplexers and registers. This multi-functionality allows a single hardware structure to replace what would traditionally require separate dedicated circuits for each operation, thereby increasing processing capacity while minimizing the increase in area and power consumption.
Solution Approach 2:
The patent employs dynamically reconfigurable data paths within the processing elements. Multiplexers and switches allow the hardware to adapt its configuration based on the current operation being performed, enabling the same physical infrastructure to support different computational tasks efficiently. This dynamic reconfiguration maximizes the utilization of existing hardware resources, improving processing capacity without proportionally increasing space requirements.
3Device complexity
If data is moved across common data busses, then hardware sharing is improved, but power usage increases and latency is introduced
Solution Approach 1:
The patent segments the data path into dedicated channels within each processing element, including separate read and write busses that operate independently. This segmentation allows data to be moved between different functional units within the PE without contending for shared resources, thereby reducing power consumption and latency while maintaining hardware efficiency. Each segment handles specific data flows, eliminating the need for arbitration and reducing overall system complexity.
Data Source
Figure 1A~1C
Figure 1D
Figure 2
AI summary
Methods and apparatus for performing machine learning tasks, and in particular, to a neural-network-processing architecture and circuits for improved handling of partial accumulation results in weight-stationary operations, such as operations occurring in compute-in-memory (CIM) processing elements (PEs). One example PE circuit for machine learning generally includes an accumulator circuit, a flip-flop array having an input coupled to an output of the accumulator circuit, a write register, and a first multiplexer having a first input coupled to an output of the write register, having a second input coupled to an output of the flip-flop array, and having an output coupled to a first input of the first accumulator circuit.