Memory and Near-Memory Die SIP for Concurrent Random Access

The patent segments the memory system into two distinct parts: a general-purpose memory die and a near-memory compute die. This segmentation allows each die to be optimized for its specific function, enabling high-performance near-memory computing for AI workloads while maintaining general-purpose memory capabilities without the overhead of integrating compute functions throughout the entire memory array.

The patent introduces an intermediary high-bandwidth interface that connects the general-purpose memory die and the near-memory compute die. This intermediary enables efficient data transfer between the two dies, achieving near-memory computing performance without requiring compute functions to be embedded within the memory array itself, thus avoiding memory area overhead.

By separating the memory die from the compute die, the patent enables independent optimization of each component. The general-purpose memory die can handle any data access pattern, while the near-memory compute die can be optimized for memory-stationary workloads. This segmentation removes dataflow restrictions because data can be freely accessed from the general-purpose memory through the high-bandwidth interface.

The general-purpose memory die provides universal data access capabilities for all types of workloads, while the near-memory compute die provides specialized high-performance computing for memory-stationary workloads. This multi-functionality approach allows the system to adapt to different workload requirements without constraining dataflow patterns.

The patent extracts the compute functions from the general-purpose memory system and places them in a separate near-memory compute die. This extraction allows the general-purpose memory to maintain high capacity while the compute die provides low-latency processing for memory-intensive AI workloads, effectively reducing the perceived latency by performing computations closer to the data.

The patent transitions from a two-dimensional planar integration to a three-dimensional stacked architecture, with the near-memory compute die stacked above the general-purpose memory die. This vertical integration reduces latency by placing compute resources in close physical proximity to the memory, enabling faster data access and processing without compromising memory capacity.

The patent segments the system into a general-purpose memory die that handles conventional data operations and a specialized near-memory compute die that optimizes for AI workloads. This segmentation allows each component to be tuned for its specific purpose, maintaining versatility for general-purpose operations while achieving high performance for memory-intensive AI tasks.

The patent applies local quality by creating a specialized near-memory compute region with optimized dataflow and processing capabilities specifically for AI workloads, while the rest of the memory system maintains general-purpose characteristics. This localized optimization enables high performance for AI tasks without compromising general-purpose functionality.