Multi-Tier Analog In-Memory Computing With Shared Converters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computing systems face inefficiencies in data shuffling between processing elements and memory, particularly for data-heavy workloads like deep neural networks, leading to high overhead and communication costs.
Innovation Solution
A multi-tier analog in-memory computing (AIMC) system with stacked tiers of resistive memory devices and shared digital-to-analog and analog-to-digital converters, utilizing a programmable logic controller to perform matrix vector multiplications in-situ, allowing for efficient handling of large matrices without external data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is shuffled between processing elements and memory in conventional computing systems, then computation can be performed, but time and energy cost increase significantly
Solution Approach 1:
The patent merges memory and processing functions into a single integrated structure where resistive memory devices serve both as storage elements and computation elements. The crossbar array integrates weight storage (in conductance values) and matrix-vector multiplication functionality, eliminating the need for separate memory and processing units and thus removing data shuffling between them.
Solution Approach 2:
The patent transitions from conventional planar computing architecture to a three-dimensional stacked crossbar architecture with multiple tiers. This vertical stacking enables simultaneous access to multiple weight matrices from different tiers, allowing parallel computation operations and reducing the time required for data access and processing.
2Productivity
If multiple tiers are stacked to handle large matrices, then processing capacity increases, but device complexity increases
Solution Approach 1:
The shared DAC and ADC units serve multiple tiers simultaneously, performing universal functions for the entire multi-tier system. The DAC converts digital input vectors to analog voltages for all tiers, while the ADC converts analog current outputs from all tiers back to digital values. This multi-functional approach reduces the number of separate components needed and simplifies the overall system architecture.
Solution Approach 2:
The programmable logic controller acts as an intermediary that manages the complex interactions between multiple tiers, DAC, ADC, and external interfaces. It coordinates the operation of shared resources, handles data routing between tiers, and manages the computation workflow, thereby reducing the complexity burden on the hardware structure itself.
3Productivity
If shared converters are used across multiple tiles, then resource utilization improves, but access overhead increases
Solution Approach 1:
The shared DAC and ADC units operate continuously without idle periods by processing inputs from multiple tiers in sequence. The programmable logic controller ensures that the converters are constantly engaged in conversion operations, eliminating wait states and maximizing the utilization of these critical resources throughout the computation process.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system reduces computing time and resource usage by performing matrix vector multiplications in-situ, minimizing the need for external memory access and enhancing processing efficiency for large datasets.
Implementation Method 1
A crossbar of resistive memory devices, including a plurality of columns, is on each tier. The crossbar is configured to encode a matrix of weights.
Implementation Method 2
A digital to analog convert (DAC) is coupled to the periphery of the first tile. The DAC is configured to encode an input vector to voltage pulses applied on the crossbar.
Implementation Method 3
The ADC is configured to measure an induced current on each column of the crossbar and digitize the induced current into a digital value.
Implementation Method 4
By encoding the Matrix parameters in the conductance of memory elements and applying voltage pulses encoding the Vector, we can exploit Ohm's and Kirchoff's laws to calculate dot products by measuring the produced currents.
Implementation Method 5
The first result and the second result are accumulated into an accumulated digital value of the first tile, represented as a counter value in a register of the ADC.
Data Source
AI summary
An analog in-memory computing (AIMC) system includes a plurality of tiles. A plurality of vertically stacked tiers are present on each tile. Each tier comprises a crossbar of resistive memory devices, configured to encode a matrix of weights. A digital to analog convert (DAC) is shared by the plurality of tiles. The DAC is configured to encode an input vector to voltage pulses applied on the crossbar. An analog to digital converter (ADC) is shared by the plurality of tiles, and includes a register of counters. The ADC is configured to measure an induced current on each column of the crossbar and digitize the induced current into a digital value. A programmable logic controller is configured to: control the ADC to retain integration values between integrations performed for each tier. An accumulation of partial integration results is performed in-situ of the tile.