Ternary In-Memory Accelerator for Energy-Efficient AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer architectures, such as the von-Neumann architecture, face bottlenecks in data transfer rates and energy consumption, particularly in data-intensive applications like big-data and AI, due to the decoupling of memory and processor, which limits the adoption of energy-efficient ternary deep neural networks (DNNs) in IoT devices.
Innovation Solution
A ternary processing cell (TPC) and circuit design that enables in-memory arithmetic operations, allowing for parallel vector-matrix multiplications within a memory array using a CMOS-based bitcell, effectively reducing energy consumption and improving performance by functioning as both a ternary storage unit and scalar multiplication unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If von-Neumann architecture with decoupled memory and processor is used, then computing power can be expanded, but data transfer bottleneck increases energy consumption and reduces throughput
Solution Approach 1:
The patent merges memory and processing functions by implementing ternary processing cells (TPCs) that perform in-memory computing. The TPC combines storage elements (first and second memory cells) with arithmetic logic, allowing multiply-accumulate operations to be performed directly within the memory array, eliminating the need for data transfer between separate memory and processor units.
Solution Approach 2:
The ternary processing cell serves multiple functions: it stores ternary values (0, 1, -1) in memory cells and simultaneously performs arithmetic operations (multiplication and accumulation). This multi-functionality allows the same hardware structure to handle both data storage and computation, reducing the energy overhead of data movement.
2Power
If von-Neumann architecture with decoupled memory and processor is used, then computing power can be expanded, but data transfer cycle time increases
Solution Approach 1:
The patent merges memory and processing functions by implementing ternary processing cells (TPCs) that perform in-memory computing. The TPC combines storage elements (first and second memory cells) with arithmetic logic, allowing multiply-accumulate operations to be performed directly within the memory array, eliminating the need for data transfer between separate memory and processor units.
3Ease of manufacture
If binary digital data is used, then hardware implementation is simple, but computational efficiency for DNNs is insufficient
Solution Approach 1:
The patent changes the numerical base parameter from binary (base-2) to ternary (base-3) representation. Ternary values (0, 1, -1) are stored using two binary memory cells per TPC, enabling more efficient representation of DNN weights and activations. This parameter change allows single-bit inputs to be processed with ternary weights, reducing the total number of operations needed for DNN inference.
4Measurement precision
If high precision is used for DNN computations, then accuracy is maintained, but energy consumption and computational cost increase
Solution Approach 1:
The patent changes the numerical base parameter from binary (base-2) to ternary (base-3) representation. Ternary values (0, 1, -1) are stored using two binary memory cells per TPC, enabling more efficient representation of DNN weights and activations. This parameter change allows single-bit inputs to be processed with ternary weights, reducing the total number of operations needed for DNN inference.
Data Source
AI summary
A circuit of cells used as a memory array and capable of in-memory arithmetic which includes a plurality of signed ternary processing, each signed ternary processing cell includes a first memory cell, adapted to hold a first digital value, a second memory cell, adapted to hold a second digital value, wherein a binary combination of the first digital value and the second digital value establishes a first signed ternary operand, a signed ternary input forming a second signed ternary operand, and a signed ternary output, wherein the signed ternary output represents a signed multiplication of the first signed ternary operand and the second signed ternary operand, a sense circuit adapted to output a subtraction result.