Partitioned Memory Architecture for In-Memory Processing Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require large memory arrays for processing, leading to increased local voltage drops and processing errors, while designers face challenges in balancing throughput and area consumption.
Innovation Solution
A partitioned memory architecture with individually selectable single or dual resistor memory elements and track-and-hold devices, allowing outputs from one processing layer to be fed back as inputs to the same structure for the next layer, minimizing IR drops and enabling structure reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large memory arrays are used to implement deep neural networks, then processing capability and throughput are improved, but local voltage drops increase leading to processing errors
Solution Approach 1:
The memory array is divided into multiple memory banks, each processing a subset of the neural network computations. This segmentation allows the system to handle large-scale processing while maintaining smaller, more reliable voltage levels within each bank, thereby reducing IR drops and processing errors.
Solution Approach 2:
Track-and-hold devices are introduced as intermediary components between memory banks and processing units. These devices buffer and stabilize voltage signals, compensating for IR drops and ensuring reliable data transmission across the memory array, thus maintaining processing accuracy in large-scale implementations.
2Productivity
If memory array size is increased to handle complex neural networks, then processing throughput is improved, but area consumption increases
Solution Approach 1:
Each memory cell is designed to perform multiple functions: storing weights, performing analog multiplication, and supporting both single-resistor and dual-resistor configurations. This multi-functionality allows the same physical array to handle complex neural networks with high throughput while minimizing the required area by eliminating redundant components.
Solution Approach 2:
The memory array supports dynamic reconfiguration between single-resistor and dual-resistor modes, allowing the system to adapt its computational density to the specific requirements of different neural network layers. This dynamic flexibility optimizes the balance between throughput and area consumption for various processing workloads.
3Reliability
If multiple processing layers are implemented using separate structures, then processing accuracy is maintained, but device complexity and area consumption increase
Solution Approach 1:
A single memory array structure is designed to support multiple processing layers by dynamically reconfiguring which memory banks are active and how they are connected. The same physical infrastructure handles weight storage, analog computation, and result accumulation for successive layers, eliminating the need for separate structures while maintaining processing accuracy through controlled signal routing.
Solution Approach 2:
The system enables continuous processing across multiple layers by maintaining active memory banks and using track-and-hold devices to preserve intermediate results between layers. This continuous operation allows outputs from one layer to be immediately fed into the next layer without requiring structural reconfiguration, reducing complexity while preserving accuracy.
Data Source
AI summary
A structure for in-memory processing includes memory banks arranged in columns and rows, each bank having bank input nodes, at least one bitline, and cells arranged in a column and connected to corresponding bank input nodes, respectively, and to the bitline(s). Each cell includes layer-specific memory elements, which are individually programmable to store layer-specific weight values and individually connectable (e.g., by switches) to the corresponding bank input node and the bitline(s). The initial memory banks in each row also include track-and-hold devices (THs) connected to the bank input nodes. For each iteration of in-memory processing, the outputs from one processing layer are feedback to pre-designated THs for use as inputs for the next processing layer, the appropriate layer-specific memory elements in the cells are connected to the corresponding bank input nodes and bitline(s), and output(s) for the next processing layer are generated.