Analog SRAM Neural Cache for In-Memory Tensor Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Recurrent neural networks face significant computational burdens due to large tensors generated, leading to output delays, which are unacceptable in applications like object detection in autonomous vehicles, and existing solutions like increasing processor bandwidth or using SRAM arrays for in-memory processing are costly or compromise network speed and efficiency.
Innovation Solution
Implementing an in-memory, analog neural network using pipelined static random access memory (SRAM) architecture that allows memory cells to store multi-bit data values, enabling analog computations and converting analog outputs to digital values for processing, thereby enhancing computational efficiency without requiring additional SRAM memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional digital SRAM memory is used for neural network processing, then data storage capacity is adequate, but computational speed and efficiency are insufficient due to frequent data transfer between storage and processing units
Solution Approach 1:
The patent replaces traditional digital binary storage mechanisms with analog memory cells that can store continuous voltage values. This substitution allows memory cells to represent multiple data values simultaneously through voltage levels, enabling in-memory computing operations that directly perform neural network calculations without transferring data between separate storage and processing units, thereby significantly improving both speed and computational efficiency
Solution Approach 2:
The patent changes the fundamental parameter of data representation from discrete binary states (0 or 1) to continuous analog voltage values. Each memory cell can store a range of voltage levels corresponding to different data values, allowing a single cell to represent multiple potential outputs. This parameter change enables parallel computation of multiple neural network values within the same memory structure, resolving the speed and efficiency contradiction
2Speed
If processor bandwidth is increased to handle large neural network tensors, then computational speed improves, but device cost and complexity increase significantly
Solution Approach 1:
The patent makes the memory structure multi-functional by enabling it to both store data and perform computational operations directly within the same analog memory cells. The memory array serves dual purposes: storing input tensors and performing matrix multiplication operations, eliminating the need for separate high-bandwidth processing units and reducing overall device complexity while maintaining high computational speed
Solution Approach 2:
The patent introduces analog voltage levels as an intermediary representation that bridges storage and computation functions. Instead of requiring high-speed digital data transfer between separate memory and processing units, the analog voltage values directly mediate both data storage and mathematical operations, reducing bandwidth requirements while maintaining computational speed
3Productivity
If SRAM arrays are used for in-memory processing, then computational efficiency improves, but the cost and hardware requirements increase
Solution Approach 1:
The patent segments the computational function across individual memory cell units, where each cell independently stores and processes data. This segmentation allows standard memory manufacturing processes to be used without requiring specialized expensive hardware, as each cell operates autonomously for computation while maintaining compatibility with existing semiconductor fabrication techniques
Solution Approach 2:
The patent enables memory cells to perform computational operations on their own without requiring external processing units. The analog memory cells self-service by directly computing neural network operations using their stored voltage values, eliminating the need for additional expensive computational hardware and reducing manufacturing costs while maintaining high computational efficiency
4Quantity of substance
If more memory cells are allocated to store neural network data, then data capacity increases, but computational density within cache memory decreases
Solution Approach 1:
The patent adds a voltage dimension to the traditional binary memory state, allowing each memory cell to represent multiple data values through different voltage levels. This dimensional change enables a single memory cell to store and compute with multiple potential outputs simultaneously, increasing both data capacity and computational density without requiring additional physical memory cells
Data Source
AI summary
Embodiments are directed to systems and methods of implementing an analog neural network using a pipelined SRAM architecture (“PISA”) circuitry disposed in on-chip processor memory circuitry. The on-chip processor memory circuitry may include processor last level cache (LLC) circuitry. One or more physical parameters, such as a stored charge or voltage, may be used to permit the generation of an in-memory analog output using a SRAM array. The generation of an in-memory analog output using only word-line and bit-line capabilities beneficially increases the computational density of the PISA circuit without increasing power requirements. Thus, the systems and methods described herein beneficially leverage the existing capabilities of on-chip SRAM processor memory circuitry to perform a relatively large number of analog vector/tensor calculations associated with execution of a neural network, such as a recurrent neural network, without burdening the processor circuitry and without significant impact to the processor power requirements.