Programmable in-memory computing accelerator for low-precision deep neural network inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-memory computing (IMC) architectures for deep neural networks face challenges in scalability, flexibility to support various layer types, and efficiency due to limited integration and inflexible data flow, leading to high latency and energy consumption.
Innovation Solution
A programmable in-memory computing accelerator (PIMCA) integrating 108 capacitive-coupling-based IMC SRAM macros with a custom instruction set architecture (ISA) and single-instruction-multiple-data (SIMD) functional units, supporting a wide range of deep neural network layers and operations, including hardware loop support.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of IMC macros are integrated on-chip to improve computation parallelism and throughput, then productivity is improved, but device complexity increases
Solution Approach 1:
The system is divided into multiple IMC processing elements (PEs), each containing a set of IMC macros that operate independently. This segmentation allows parallel processing while managing complexity through modular organization, enabling the integration of 108 IMC macros without overwhelming system complexity.
Solution Approach 2:
A custom instruction set architecture (ISA) is developed that provides universal support for multiple DNN layer types including convolution, fully-connected, batch normalization, and activation layers. This multi-functional ISA allows the same hardware architecture to handle diverse computational tasks, improving productivity without requiring separate specialized hardware for each layer type.
2Productivity
If hardware loop support is added to reduce latency and instruction counts, then productivity is improved, but device complexity increases
Solution Approach 1:
Hardware loop support is pre-integrated into the custom ISA, allowing loop operations to be executed directly in hardware without software interpretation overhead. This preliminary preparation of loop structures reduces execution latency while the hardware-based implementation keeps the added complexity minimal compared to software alternatives.
3Loss of energy
If off-chip data movement is eliminated through in-memory computing, then energy efficiency is improved, but manufacturing precision requirements increase
Solution Approach 1:
The IMC macros merge memory storage and computation functions into a single integrated unit, eliminating the need for separate memory and processing components. This merging allows data to be processed in-place without movement between memory and processor, dramatically reducing energy consumption while the capacitive coupling mechanism provides sufficient precision for low-precision DNN inference.
4Device complexity
If the data flow is hard-wired to support only specific layer types, then device complexity is reduced, but adaptability deteriorates
Solution Approach 1:
The custom ISA implements dynamic data flow paths that can be reconfigured at runtime to support different DNN layer types. Instead of fixed hard-wired connections, the system uses programmable control signals to dynamically route data and configure operational modes, enabling support for convolution, fully-connected, batch normalization, and activation layers without increasing physical hardware complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The PIMCA achieves a peak throughput of 4.9 tera operations per second and system-level energy efficiency of 437 TOPS/W, significantly reducing latency and energy consumption by eliminating off-chip data movement and optimizing instruction counts.
Implementation Method 1
A programmable in-memory computing (IMC) accelerator for low-precision deep neural network inference, also referred to as PIMCA, integrates a large number of capacitive-coupling-based IMC static random-access memory (SRAM) macros
Data Source
AI summary
A programmable in-memory computing (IMC) accelerator for low-precision deep neural network inference, also referred to as PIMCA, is provided. Embodiments of the PIMCA integrate a large number of capacitive-coupling-based IMC static random-access memory (SRAM) macros and demonstrate large-scale integration of IMC SRAM macros. For example, a 28 nm prototype integrates 108 capacitive-coupling-based IMC SRAM macros of a total size of 3.4 megabytes (Mb), demonstrating one of the largest IMC hardware to date. In addition, a custom instruction set architecture (ISA) is developed featuring IMC and single-instruction-multiple-data (SIMD) functional units with hardware loop to support a range of deep neural network (DNN) layer types. The 28 nm prototype chip achieves a peak throughput of 4.9 tera operations per second (TOPS) and system-level peak energy-efficiency of 437 TOPS per watt (TOPS/W) at 40 megahertz (MHz) with a 1 volt (V) supply.