3D Stacked SRAM Compute-in-Memory Architecture

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The increasing size of models in AI accelerators leads to higher IC fabrication costs and scalability issues, as well as challenges in memory density, data movement, energy consumption, and latency.

Innovation Solution

A computing device with a 3D stacked architecture, featuring arrays of compute units and routers on multiple substrates, which enables efficient data transmission through both horizontal and vertical routing, utilizing compute-in-memory modules for vector-matrix multiplications and local updates.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If 2.5D scaling or multilayer stacking is used to increase memory density, then memory density and compute power are improved, but manufacturing complexity, alignment precision requirements, and cost increase

Engineering Contradiction:

Improvememory densityVSAvoidmanufacturing complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent transitions from 2D planar integration to 3D vertical stacking by implementing multiple layers of compute units and memory structures stacked along the vertical dimension. This enables memory density and compute power to increase without proportionally increasing the horizontal footprint, while the modular layer design simplifies manufacturing compared to monolithic 3D structures.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system is divided into discrete modular layers, with each layer containing compute units, memory structures, and interconnect structures. These modular layers can be independently fabricated and then stacked, reducing manufacturing complexity and alignment precision requirements compared to monolithic integration.

Inventive Principle:

Principle #1Segmentation

2Quantity of substance

If larger ICs are used to store more parameters, then model capacity is improved, but fabrication cost increases quadratically

Engineering Contradiction:

Improvenumber of parametersVSAvoidfabrication cost

Core Design Contradiction:

Quantity of substanceVSEase of manufacture

Solution Approach 1:

The patent utilizes vertical stacking to increase the number of storable parameters without proportionally increasing the horizontal IC area. By stacking multiple layers of memory and compute units, the system achieves higher parameter capacity while maintaining a compact footprint, thereby avoiding quadratic cost increases associated with larger planar ICs.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system employs a composite architecture integrating multiple material systems and structural layers, including conductive interconnects, dielectric layers, and active device layers. This composite approach enables high parameter density while maintaining manufacturing efficiency through standardized fabrication processes for each material layer.

Inventive Principle:

Principle #40Composite materials

3Adaptability or versatility

If data is moved between separate memory and compute units, then architectural flexibility is improved, but energy consumption and latency increase

Engineering Contradiction:

Improvearchitectural flexibilityVSAvoidenergy consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent merges memory structures and compute units into integrated compute-in-memory blocks where memory and processing are co-located within the same layer or adjacent layers. This integration eliminates or minimizes data movement between separate memory and compute units, reducing energy consumption and latency while maintaining architectural flexibility through configurable interconnects.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system introduces intermediate buffer structures and local interconnect networks that facilitate efficient data exchange between memory and compute units. These intermediary structures are optimized for low-latency, low-energy communication, enabling flexible data flow patterns without the energy penalty of long-distance data movement.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250046350A1Methods for efficient 3D SRAM-based compute-in-memory

Publication Date: 2025.02.06 RAIN NEUROMORPHICS INC

AI summary

A computing device is described. The computing device includes first and second arrays of compute units and first and second arrays of routers. The first array of compute units is arranged on a first substrate and includes a first plurality of compute-in-memory (CIM) modules. The first array of routers is configured to route information horizontally among the first array of compute units. The second array of compute units is arranged on a second substrate and includes a second plurality of CIM modules. The second substrate is disposed vertically from the first substrate. The second array of routers is configured to route the information horizontally among the second array of compute units on the second substrate. The first array of routers and the second array of routers send the information vertically between the first substrate and the second substrate.