Query Controller in Stacked-Die Memory for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Stacked-die memory devices face bandwidth and latency limitations due to interconnect bottlenecks, leading to inefficiencies in processing systems, particularly when performing multiple memory accesses, which incur high power consumption and reduced processing efficiency.
Innovation Solution
Implementing a stacked-die memory device with an integrated query controller that performs query operations directly on memory cell circuitry, utilizing through-silicon vias for efficient data access and offloading processing tasks from external devices, allowing the system to function as a key-value store or content-addressable memory, thereby reducing latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If stacked-die memory is used to improve bandwidth and reduce intra-device latency, then memory performance is improved, but inter-device bandwidth and latency become bottleneck due to separate implementation of system memory and other components
Solution Approach 1:
The patent combines the stacked-die memory with a query controller into a single integrated device. The query controller is coupled to the memory cell circuitry through the same high-speed interconnect (e.g., TSVs) that connects memory dies, allowing the controller to access memory at full bandwidth without external interconnect limitations. This merging resolves the contradiction by enabling both fast memory access and efficient query processing within the same device package.
2Loss of information
If multiple memory accesses are performed for query operations, then data can be retrieved, but bandwidth and latency penalties are incurred twice for each access due to back-and-forth communication
Solution Approach 1:
The query controller performs query operations (search, insert, write, delete) directly on the stacked-die memory without requiring external device intervention. The controller is self-sufficient in executing complex memory operations, reducing the need for repeated back-and-forth communication with external processors. This self-service capability reduces both the bandwidth penalties and power consumption associated with multiple external accesses.
3Ease of operation
If query operations are performed externally, then memory can be accessed, but processing efficiency decreases due to handshaking process and propagation delay
Solution Approach 1:
The query controller is integrated within the same three-dimensional stacked-die structure, utilizing the vertical stacking dimension to place the controller in close proximity to the memory cell circuitry. This spatial integration within the stack eliminates the need for long horizontal interconnects and handshaking protocols required by external devices, significantly reducing propagation delay and latency while maintaining ease of operation.
Data Source
AI summary
An integrated circuit (IC) package includes a stacked-die memory device. The stacked-die memory device includes a set of one or more stacked memory dies implementing memory cell circuitry. The stacked-die memory device further includes a set of one or more logic dies electrically coupled to the memory cell circuitry. The set of one or more logic dies includes a query controller and a memory controller. The memory controller is coupleable to at least one device external to the stacked-die memory device. The query controller is to perform a query operation on data stored in the memory cell circuitry responsive to a query command received from the external device.