Bit-serial PISA SRAM for Neural Network In-Memory Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Recurrent neural networks face significant mathematical burdens due to large tensors generated, leading to output delays, which are unacceptable in applications like object detection in autonomous vehicles, and existing solutions like increasing clock speeds or adding processing cores are costly and increase device size/power consumption.
Innovation Solution
Implementing a pipelined SRAM architecture (bit-serial PISA) in on-chip processor memory for in-memory processing of recurrent neural networks, preloading models and weights into high-bandwidth memory locations to minimize off-chip data transfers and enhance responsiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred from storage to system memory for processing, then data transfer rate is improved, but data transfer time and system complexity increase
Solution Approach 1:
The patent combines storage memory and processing units into an integrated memory processing system where computation is performed directly within the memory array. This merging eliminates the need for separate data transfer between storage and processing units, thereby improving effective data transfer rate while reducing actual data movement time.
Solution Approach 2:
The patent introduces a memory controller as an intermediary that manages data flow and coordinates processing operations within the memory array. This mediator optimizes data access patterns and enables in-memory computation without requiring traditional data movement to external processors, reducing transfer time while maintaining high effective throughput.
2Speed
If clock speed is increased to improve processing speed, then neural network processing speed is improved, but device cost and power consumption increase
Solution Approach 1:
The patent replaces traditional mechanical/CPU-based processing with in-memory computing using content-addressable memory and associative processing. This substitution performs neural network operations directly within the memory array using parallel bit-level operations, achieving high processing speed without the power consumption penalties of increasing CPU clock speeds.
Solution Approach 2:
The patent changes the fundamental operating parameters of the processing system by performing computations at the bit level in parallel across the entire memory array, rather than sequentially at higher bit levels. This parameter change enables massive parallelism that achieves high throughput with lower power consumption per operation.
3Productivity
If processing cores are added to improve computational capacity, then neural network processing capability is improved, but device size and cost increase
Solution Approach 1:
The patent makes the memory array universal by enabling it to perform multiple functions: data storage, data processing, and neural network computation. This multi-functionality eliminates the need for separate processing cores, achieving high computational capacity while minimizing device area by using the same physical structure for both storage and processing.
Solution Approach 2:
The patent transitions from traditional von Neumann architecture with separate processing and storage dimensions to an integrated memory processing architecture where computation occurs within the memory array itself. This dimensional integration achieves massive parallelism and high computational capacity without proportionally increasing device area.
Data Source
AI summary
The present disclosure is directed to systems and methods of implementing a neural network using in-memory, bit-serial, mathematical operations performed by a pipelined SRAM architecture (bit-serial PISA) circuitry disposed in on-chip processor memory circuitry. The on-chip processor memory circuitry may include processor last level cache (LLC) circuitry. The bit-serial PISA circuitry is coupled to PISA memory circuitry via a relatively high-bandwidth connection to beneficially facilitate the storage and retrieval of layer weights by the bit-serial PISA circuitry during execution. Direct memory access (DMA) circuitry transfers the neural network model and input data from system memory to the bit-serial PISA memory and also transfers output data from the PISA memory circuitry to system memory circuitry. Thus, the systems and methods described herein beneficially leverage the on-chip processor memory circuitry to perform a relatively large number of vector/tensor calculations without burdening the processor circuitry.