In-Memory Compute Array with Dynamic Column Multiplexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional learning/inference machines face inefficiencies in memory utilization due to varying data sets across different layers of neural networks, leading to suboptimal aspect ratio utilization and potential accuracy loss.
Innovation Solution
A memory array architecture that employs configurable multiplexers and sensing circuits to optimize memory utilization by dynamically adjusting compute-cluster cycles based on varying data sets, using cluster cycle management circuitry to manage multiplexers and sensing circuits for efficient data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional memory arrays are used with fixed architecture, then device complexity is reduced, but memory utilization efficiency deteriorates due to varying data sets across different neural network layers
Solution Approach 1:
The memory array architecture implements dynamic reconfiguration of column multiplexers and sensing circuits based on the aspect ratio requirements of different neural network layers. The system adjusts the number of active columns and their arrangement dynamically during inference operations, transforming a static memory structure into an adaptive one that optimizes for each layer's specific data requirements.
Solution Approach 2:
The memory array is divided into multiple column groups that can be independently configured and activated. Each column group can be selectively enabled or disabled based on the computational requirements of the current layer, allowing the system to partition memory resources efficiently and activate only the necessary segments for each operation.
2Productivity
If memory array is reconfigured for different aspect ratios, then memory utilization improves, but compute-cluster cycle time increases due to reconfiguration overhead
Solution Approach 1:
The system pre-configures multiple column multiplexer settings and sensing circuit arrangements corresponding to different layer types before inference begins. During operation, pre-computed aspect ratio requirements are used to directly select from pre-prepared configurations, avoiding real-time reconfiguration and minimizing cycle time extension.
Solution Approach 2:
The memory array is reconfigured periodically at layer boundaries rather than continuously during operations. Each reconfiguration event is synchronized with the transition between neural network layers, creating a rhythmic pattern of configuration changes that minimizes interference with ongoing computations and allows for optimized timing.
3Reliability
If fixed memory architecture is used, then device complexity is low, but accuracy is lost due to suboptimal aspect ratio utilization
Solution Approach 1:
Different column groups within the memory array are assigned specific aspect ratio optimizations tailored to particular layer types. Each local region of the memory array can be independently optimized for its intended computational task, with dedicated column configurations that match the specific accuracy requirements of different neural network layers.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A memory array arranged in multiple columns and rows. Computation circuits that each calculate a computation value from cell values in a corresponding column. A column multiplexer cycles through multiple data lines that each corresponds to a computation circuit. Cluster cycle management circuitry determines a number of multiplexer cycles based on a number of columns storing data of a compute cluster. A sensing circuit obtains the computation values from the computation circuits via the column multiplexer as the column multiplexer cycles through the data lines. The sensing circuit combines the obtained computation values over the determined number of multiplexer cycles. A first clock may initiate the multiplexer to cycle through its data lines for the determined number of multiplexer cycles, and a second clock may initiate each individual cycle. The multiplexer or additional circuitry may be utilized to modify the order in which data is written to the columns.