Utilization-Based Tile Mapping for 3D Analog In-Memory Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In 3D Analog In-Memory Computing (AIMC) systems, the unique resource constraint of tiles, where only a single tier can be active at a time, makes parallel computation and model pipelining challenging due to varying execution times across neural network layers.
Innovation Solution
A computer program product and method that balances the utilization of tiles in an AIMC system by identifying tiles, mapping neural network layers, determining operation counts, and assigning layers to tiles to achieve an equalized utilization rate across tiles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If sequential layer mapping is used to minimize maximum tile latency, then latency is reduced, but tile utilization becomes unbalanced and parallelism is limited
Solution Approach 1:
The patent implements dynamic tile utilization balancing by allowing the system to adaptively assign layers to tiles based on real-time utilization metrics. The mapping is not fixed but can be reconfigured to maintain balanced utilization across tiles, enabling the system to dynamically respond to varying workload demands and achieve both low latency and high productivity
Solution Approach 2:
The system changes the mapping parameters between layers and tiles to optimize performance. By adjusting which layers are assigned to which tiles based on utilization rates, the system can transform the static sequential mapping into a flexible configuration that balances latency reduction with tile utilization equilibrium, enabling parallel processing of multiple models
2Quantity of substance
If weight stationary architecture is used for in-memory compute, then memory capacity is increased, but data shuffling between memory and processing unit is eliminated at the cost of reduced flexibility
Solution Approach 1:
The patent segments the neural network into multiple layers that can be distributed across multiple tiles. This segmentation allows the weight stationary architecture to maintain large memory capacity while enabling flexible parallel processing of different layer segments, thus preserving adaptability despite the in-memory constraint
Solution Approach 2:
The tile architecture is designed to be universal and multi-functional, capable of processing different layers and supporting multiple neural network models. This universality allows the weight stationary architecture to maintain both high memory capacity and computational flexibility by enabling tiles to perform various computational tasks across different models and layers
3Device complexity
If only single tier is active at a time in 3D AIMC tile, then resource constraint is simplified, but parallel computation and model pipelining become challenging
Solution Approach 1:
The patent exploits the 3D vertical dimension of the memory architecture to enable parallel computation. By organizing layers vertically across multiple tiers and activating different tiers in parallel for different models, the system transforms the single-tier constraint into a multi-dimensional processing capability, maintaining simple resource management while achieving high productivity through vertical parallelism
Data Source
AI summary
A method for balancing utilization of tiles in an analog in-memory computing system includes identifying, by a computer processor, a plurality of tiles in the analog in-memory computing system. The computer processor receives a plurality of layers in a neural network being processed by the analog in-memory computing system. The computer processor maps the plurality of layers in the neural network to the plurality of tiles. The computer processor determines a number of operations for each of the tiles in the plurality of tiles. The computer processor determines an equalized utilization rate for the tiles in the plurality of tiles. In addition, the computer processor assigns the layers to the plurality of tiles. The tiles are assigned so that a first utilization rate of a first tile is balanced relative to a second utilization rate of a second tile in the analog in-memory computing system.