In-Memory Computing Module Layer-Symmetric Design Memory Wall
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The 'Memory Wall' phenomenon, where processor performance outpaces memory performance, leading to significant latency and bandwidth constraints in high-performance computing, due to the asynchronous development of processor and memory technologies, necessitates an innovative solution to integrate memory and computing units closely to enhance access speed and bandwidth.
Innovation Solution
An in-memory computing module with a layer-symmetric design comprising multiple computing submodules, each containing a computing unit, memory units, and a routing unit, connected via bonding connections to achieve low latency and high data bandwidth, allowing direct or indirect access to memory units across submodules, and enabling flexible storage capacity customization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional separate computing units and memory units are used, then device complexity is reduced and ease of manufacture is improved, but memory access latency increases and computing performance deteriorates
Solution Approach 1:
The patent combines memory units and computing units into an integrated in-memory computing structure where memory units are directly coupled with computing units through bonding connections. This merging eliminates the need for separate data retrieval and writing operations, significantly reducing memory access latency while maintaining manageable device complexity through modular design.
Solution Approach 2:
The patent employs 3D stacking technology to vertically integrate multiple layers of memory units and computing units. By transitioning from a 2D planar layout to a 3D stacked architecture, the system reduces data transmission distance and access latency while organizing complex components in a structured manner that facilitates manufacturing.
2Productivity
If processor performance is continuously improved, then computing capacity increases, but memory access speed becomes a bottleneck and productivity decreases
Solution Approach 1:
By integrating memory units directly with computing units through bonding connections, the patent enables high-speed data exchange between memory and computing components. This combination eliminates the memory wall bottleneck, allowing processor performance improvements to translate directly into computing productivity without being constrained by memory access speed.
3Loss of time
If 3D stacking technology with TSV is used to integrate memory and computing, then memory access latency is reduced, but manufacturing complexity increases and reliability decreases
Solution Approach 1:
The patent segments the integration process into two distinct stages: first, fabricating memory units and computing units separately on different wafers using mature and reliable processes; second, bonding these pre-fabricated units together through wafer-level bonding. This segmentation avoids the reliability issues of deep TSV fabrication while achieving low latency through 3D stacking.
Solution Approach 2:
The patent performs preliminary fabrication of memory units and computing units on separate wafers before bonding them together. This preliminary action allows each component to be optimized and tested independently, ensuring high reliability before integration, while the subsequent bonding process establishes the low-latency 3D connected architecture.
4Productivity
If memory and computing units are integrated in the same chip, then memory access latency is reduced and computing performance is improved, but device complexity increases
Solution Approach 1:
The patent divides the in-memory computing system into modular units: memory units, computing units, and bonding connection layers. Each module is fabricated separately using standard processes, then assembled through wafer-level bonding. This segmentation reduces integration complexity while achieving high computing performance through close proximity of memory and computing units.
Data Source
AI summary
The present application relates to an in-memory computing module and method, and an in-memory computing network and a construction method therefor. The in-memory computing module comprises at least two computing submodules, and low latency can be achieved when computing units in the computing submodules access memory units. Multiple computing submodules present a symmetric layer design, and such a symmetric layer structure facilitates the construction of a topology network so as to achieve large-scale or ultra-large-scale computation. The memory capacity of the memory units in each computing submodule can be customized, and designed flexibly. These computing submodules are in a bonding connection, and the data bit width after the bonding connection may be positive integer multiple of the data bit width of the computing units, so that high data bandwidth is achieved. The in-memory computing network uses the in-memory computing module, so that the requirements for different scales of computation can be satisfied.