On-Die Network Switches for Vertical Memory Stack Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In vertically stacked memory dies, increased memory array banks and dies lead to higher latency and inefficient routing due to increased interconnect sharing, congestion, and link failures, as well as area constraints and inefficient network logic in 3D integrated circuits.
Innovation Solution
Implementing on-die network switches with programmable routing tables that use target addresses to determine routes through vertically and horizontally stacked memory dies, avoiding congested or failed links by using vertical and horizontal hops, and distributing network switches across memory dies to manage traffic efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the number of memory array banks per memory die and vertically stacked memory dies are increased to provide higher memory storage capacity, then memory storage capacity is improved, but routing latency increases and interconnect sharing efficiency deteriorates
Solution Approach 1:
The patent segments the monolithic routing logic into distributed network switches placed on individual memory dies. Each switch independently handles routing decisions for its local die, dividing the routing function across multiple spatial locations. This segmentation reduces contention and latency by allowing parallel routing decisions rather than centralized sequential processing.
Solution Approach 2:
The patent introduces horizontal routing paths between memory dies in addition to vertical paths through TSVs. This creates a two-dimensional routing topology (horizontal and vertical dimensions) rather than purely vertical one-dimensional routing. Requests can now travel horizontally across the memory stack to reach target banks, providing alternative paths that reduce latency and avoid congested vertical interconnects.
2Quantity of substance
If more memory dies are added to the vertical stack to increase storage capacity, then memory storage capacity is improved, but the chance of link failures increases
Solution Approach 1:
The patent implements dynamic routing where network switches can adaptively select alternative paths based on real-time link status. When a vertical TSV or horizontal interconnect fails, the routing logic dynamically reroutes requests through available paths. This dynamic adaptation maintains system reliability despite increased link count in larger memory stacks.
Solution Approach 2:
The patent designs redundant routing paths in advance before failures occur. Multiple horizontal and vertical paths are provisioned throughout the memory stack, creating a cushion of alternative routes. When link failures happen, these pre-established alternative paths are already available to maintain connectivity, preventing system failure rather than reacting afterward.
3Ease of operation
If through silicon vias and their corresponding keep out zones are used for interconnects between processor die and memory dies, then vertical interconnect functionality is achieved, but on-die area for active devices and signal routes is reduced
Solution Approach 1:
The patent makes memory dies serve dual functions: they provide memory storage capacity while also acting as routing nodes in the interconnect network. By placing network switches on memory dies, the dies participate in both data storage and data routing functions. This multi-functionality reduces the need for separate dedicated routing infrastructure, optimizing area utilization.
4Adaptability or versatility
If an interconnection network with network logic is used in place of direct path bus wires, then routing flexibility is improved, but routing efficiency deteriorates due to end-to-end static routing in host processor
Solution Approach 1:
The patent introduces on-die network switches as intermediary routing nodes distributed across memory dies. These intermediaries perform local routing decisions at the point of destination rather than relying solely on end-to-end routing from the host processor. This distributed intermediary approach maintains routing flexibility while improving efficiency by reducing the burden on host processor logic and enabling localized optimization.
Data Source
AI summary
Systems, apparatuses, and methods for routing traffic through vertically stacked memory are disclosed. A computing system includes a host processor die and multiple vertically stacked memory dies. The host processor die generates memory access requests for the data stored in the multiple memory array banks in the memory dies. At least one memory die uses an on-die network switch with a programmable routing table for routing packets corresponding to the generated memory requests. Routes use both vertical hops and horizontal hops to reach the target memory array bank and to avoid any congested or failed resources along the route. The vertically stacked memory dies use through silicon via interconnects and at least one via does not traverse through all of the memory dies. Accordingly, the host processor die does not have a direct connection to one or more of the multiple memory dies.