Memory Device Address Generation for In-Memory MAC Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The processing performance of neural networks is limited by the performance of multiply and accumulate (MAC) operations, which are often bottlenecked by data movement between the central processing unit and memory, especially in memory-intensive applications.
Innovation Solution
A memory device with an integrated address generator and processing unit that generates target addresses internally by adding offsets to a base address, allowing operations like MAC to be performed directly in the memory, reducing the need for external address computation and alignment issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If MAC operations are performed using external processing units with separate memory, then data movement between CPU and memory causes bottlenecks, but integrating processing within memory increases device complexity
Solution Approach 1:
The patent combines the processing unit with the memory device to form an integrated PIM architecture. The processing unit is directly coupled to the memory array, allowing MAC operations to be performed within the memory device itself, eliminating the need for separate external processing units and reducing data movement bottlenecks between CPU and memory.
Solution Approach 2:
The address generator is divided into multiple counters (first counter for data storage instructions, second counter for processing operations, third counter for different memory regions) that can be independently controlled. This segmentation allows flexible address generation for different operation types while maintaining a unified integrated memory-processing architecture.
2Loss of time
If target addresses are generated externally by the host, then address generation delays occur, but internal address generation increases control complexity
Solution Approach 1:
The memory device performs self-service by generating its own target addresses internally using the address generator. The address generator receives a base address from the host and automatically generates subsequent target addresses by adding offsets, eliminating the need for the host to generate each address individually and reducing address generation delays.
Solution Approach 2:
The base address is provided in advance by the host, and the address generator pre-calculates target addresses by adding offsets before actual memory operations occur. This preliminary address generation prepares the memory system for efficient sequential access patterns without waiting for external address computation during operation execution.
3Ease of operation
If data is stored and accessed using sequential addresses from the host, then alignment issues occur, but internal offset-based address generation requires additional counter mechanisms
Solution Approach 1:
The address generator acts as an intermediary between the host and the memory array. It receives the base address from the host and uses counters as intermediate components to generate properly aligned target addresses by adding offsets. This intermediary function handles address alignment internally, shielding the host from alignment complexity while ensuring correct memory access patterns.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A memory device includes a memory array, an address generator, a data register, and a processing unit. The address generator is configured to receive an instruction and a base address of the instruction from a host, and sequentially generate target addresses for performing operations of the instruction by sequentially adding offsets to the base address. The data register is configured to store data values corresponding to one or more of the target addresses. The processing unit is configured to perform one or more of the operations of the instruction based on the data values.