Memory-Logic Die Signal Routing for Mode-Based High-Bandwidth Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory devices face inefficiencies in data transfer between memory and logic dies, particularly due to reliance on traditional I/O interfaces and global data buses, which hinder high-bandwidth data exchange necessary for advanced operations like deep learning and AI processing.
Innovation Solution
Implementing a wide bus for direct data transfer between memory and logic dies via wafer-on-wafer bonding, utilizing transceivers located on either die to facilitate efficient signal routing without traditional I/O interfaces, enabling high-bandwidth data transfer and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional I/O interfaces and global data buses are used for data transfer between memory and logic dies, then device compatibility and standardization are maintained, but data transfer bandwidth and processing speed are limited
Solution Approach 1:
The patent extracts the data transfer function from traditional I/O interfaces and global data buses, creating a dedicated direct transfer path between memory die and logic die. This removes the bottleneck of conventional interfaces while maintaining compatibility through selective activation of transfer modes.
Solution Approach 2:
The patent implements dynamic mode switching between traditional I/O mode and direct transfer mode. The system can adaptively select the appropriate transfer mode based on operational requirements, enabling both high bandwidth direct transfer and standard compatibility when needed.
2Speed
If direct data transfer path is implemented between memory and logic dies, then data transfer speed and bandwidth are improved, but power consumption increases due to dedicated transfer circuits
Solution Approach 1:
The system dynamically switches between transfer modes based on operational needs. During AI/ML operations requiring high bandwidth, the direct transfer path is activated for optimal speed. For standard operations, the traditional I/O path is used to minimize power consumption, achieving energy-efficient adaptive performance.
3Productivity
If mode-based operations are implemented for AI/ML processing, then processing efficiency for specific workloads is improved, but control logic complexity increases
Solution Approach 1:
The memory device is designed with multi-functional capability to support both traditional memory operations and AI/ML-specific operations. By integrating multiple operational modes within a single device architecture, the system achieves workload-optimized performance without requiring separate dedicated hardware for each function.
Data Source
AI summary
A memory device includes a memory die bonded to a logic die. A logic die that is bonded to a memory die via a wafer-on-wafer bonding process can receive signals indicative of input data from a global data bus of the memory die and through a bond of the logic die and memory die. The logic die can also receive signals indicative of kernel data from local input/output (LIO) lines of the memory die and through the bond. The logic die can perform a plurality of operations at a plurality of vector-vector (VV) units utilizing the signals indicative of input data and the signals indicative of kernel data. The inputs and the outputs to the VV units can be configured based on a mode of the logic die.