Memory Processing Unit with Multi-Accumulator MAC for Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computing systems face challenges in reducing processing latency and power consumption due to the time-consuming transfer of large data sets between memory and processing units.
Innovation Solution
The implementation of a memory processing unit (MPU) with interleaved memory and processing regions, where multi-accumulator multiply-and-accumulate (MAC) units perform matrix dot products with time division multiplexing and data reuse, allowing compute cores to compute output feature map values simultaneously without reloading weight values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred from memory to processing units for computation, then processing can be performed, but processing latency and power consumption increase
Solution Approach 1:
The patent merges memory and processing units into a unified memory processing unit (MPU) where compute cores are integrated within or adjacent to memory regions. This integration allows data to be processed in-place within the memory structure itself, eliminating the need for separate data transfers between memory and processing units, thereby reducing processing latency and improving overall productivity
Solution Approach 2:
The MPU is segmented into multiple compute cores distributed across different memory regions, with each compute core capable of independent operation. This segmentation allows parallel processing of different data elements simultaneously, improving throughput while maintaining low latency through localized data access within each compute core's memory region
2Productivity
If data is transferred from memory to processing units for computation, then processing can be performed, but power consumption increases
Solution Approach 1:
By combining memory and processing functions into a single integrated unit, the patent eliminates repeated data transfers between memory and processing units. This integration significantly reduces the energy consumption associated with data movement, as the compute cores can perform computations directly within the memory structure without requiring high-power data transfer operations
Solution Approach 2:
The memory regions serve dual purposes as both storage and computation targets, allowing the system to process data in-place without requiring external data transfers. This self-service capability reduces the energy burden on data transfer mechanisms while maintaining processing functionality
3Productivity
If weight values are reloaded for each computation, then computation can be performed, but processing time increases
Solution Approach 1:
The patent implements preliminary action by loading weight values into registers or buffers within the compute cores before performing matrix dot product operations. This allows the compute cores to reuse weight values across multiple computation cycles without repeatedly accessing memory, significantly reducing computation time and improving throughput
Solution Approach 2:
The system recovers and reuses weight values that would otherwise be discarded by implementing caching mechanisms within the compute cores. Weight values are maintained in registers or local buffers and reused across multiple operations, eliminating the need for continuous memory reloading and reducing overall computation time
Data Source
AI summary
A multi-accumulator multiply-and-accumulate (MAC) unit can include a multiplier and a plurality of accumulators. The multiplier can be configured to multiply a given element of a corresponding column of a first matrix and a plurality of elements of a corresponding row of a second matrix to generate a plurality of corresponding partial product elements that can be accumulated by corresponding ones of the plurality of accumulators.