Row-by-Row CNN In-Memory Compute Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital hardware struggles with poor array utilization and significant re-arrangement of activations in convolutional neural networks (CNNs) when mapped onto in-memory compute fabrics like analog crossbar arrays, leading to inefficient throughput.
Innovation Solution
A row-by-row convolution neural network scheme is implemented using an in-memory compute architecture, which includes a controller, filter memory, multiply-accumulate crossbar arrays, capacitors, steering circuits, comparator circuits, and peripheral circuitry to manage the generation of output images and perform parallel computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If CNN weights are mapped onto in-memory compute fabrics like analog crossbar arrays, then weight re-use is improved, but array utilization deteriorates and significant re-arrangement of activations is required
Solution Approach 1:
The patent segments the computation process into row-by-row processing stages, where each stage handles a specific row of the input image. This segmentation allows the system to process data in manageable chunks that can be efficiently mapped to the crossbar array, improving both array utilization and weight re-use without requiring extensive re-arrangement of activations between stages.
Solution Approach 2:
The patent applies preliminary action by pre-organizing the input activations and weight matrices into a format that is optimized for row-by-row processing before they are fed into the crossbar array. This pre-processing step ensures that when data enters the compute fabric, it is already in the optimal configuration, eliminating the need for time-consuming re-arrangement operations during computation and maximizing array utilization.
2Ease of manufacture
If naive mapping of CNN weights onto analog crossbar arrays is performed, then implementation simplicity is improved, but throughput deteriorates due to poor array utilization and significant re-arrangement of activations
Solution Approach 1:
The patent introduces dynamic control mechanisms that adapt the data flow and computation schedule based on the specific requirements of each convolution operation. This dynamic approach allows the system to maintain simple hardware architecture while achieving high throughput by flexibly managing when and how data is processed, eliminating the need for complex static re-arrangement structures.
Solution Approach 2:
The patent ensures continuity of useful action by designing a pipeline architecture where multiple rows are processed in continuous sequence without idle periods. The row-by-row processing approach allows each stage of the crossbar array to remain continuously occupied with productive work, maximizing throughput while maintaining implementation simplicity through regular, predictable data flow patterns.
3Productivity
If multiple copies of the same neural network weights are used for row-by-row processing, then throughput is improved, but memory requirements increase
Solution Approach 1:
The patent applies the nested doll principle by organizing multiple weight copies in a hierarchical memory structure where weights are nested across different memory levels. Instead of duplicating full weight matrices in separate memory blocks, the system nests weight copies within a unified memory architecture, allowing efficient sharing and reuse of weight data across multiple processing rows while minimizing total memory requirements.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach achieves high throughput and low energy consumption with minimal digital processing, suitable for edge computing hardware in image classification and other tasks where CNNs are used.
Implementation Method 1
a plurality of multiply-accumulate crossbar arrays coupled to the filter memory and configured for the parallel computation of elements of the given row for each of the plurality of output images
Implementation Method 2
a bank of capacitors coupled to the plurality of multiply-accumulate crossbar arrays
Implementation Method 3
a plurality of sets of steering circuits coupled to the bank of capacitors and configured to steer currents generated by the plurality of multiply-accumulate crossbar arrays to corresponding capacitors
Implementation Method 4
a plurality of sets of comparator circuits coupled to the bank of capacitors and configured to pulse-width modulate a signal based on a voltage of a corresponding capacitor
Data Source
AI summary
A system for implementing a row-by-row convolution neural network using an in-memory compute architecture. A controller is configured to manage generation of a plurality of output images. A filter memory is configured to store copies of each of a plurality of sets of image filters. A plurality of multiply-accumulate crossbar arrays is configured for the parallel computation of elements of the given row for each of the plurality of output images. A plurality of sets of steering circuits is coupled to a bank of capacitors and configured to steer currents generated by the plurality of multiply-accumulate crossbar arrays to corresponding capacitors of the bank of capacitors. A plurality of sets of comparator circuits are configured to pulse-width modulate a signal based on a voltage of a corresponding capacitor of the bank of capacitors. Peripheral circuitry is configured to output elements of the plurality of output images via pulse-width modulated signals.