Integrated GPU/AI Core With CBA NAND and HBM Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional processing cores, including GPUs and AI processors, face challenges in balancing memory capacity, bandwidth, and power requirements, as non-volatile memories like NAND dies provide ample capacity but low bandwidth, while volatile memories like DRAM offer high bandwidth but limited capacity, and existing solutions fail to meet the specialized needs of these advanced processors.
Innovation Solution
Integration of a processor directly onto a high-capacity, high-bandwidth non-volatile CBA memory tile, with vertical passthrough zones, allowing direct data transfer, and the use of an interposer to connect the processor directly to a CBA memory tile, which includes CMOS logic circuits and high bandwidth memory (HBM) stacks, facilitating direct data transfer through fine-pitch through silicon vias.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If non-volatile memories like NAND dies are used, then memory capacity is improved, but bandwidth is reduced
Solution Approach 1:
The patent combines non-volatile NAND memory dies with volatile HBM memory dies into a single integrated memory package. The NAND dies provide high capacity storage while the HBM dies provide high bandwidth access, and they are merged through a shared package with integrated interconnects to deliver both capacity and speed simultaneously.
Solution Approach 2:
The patent introduces an intermediary controller that manages data transfer between the processor, NAND memory, and HBM memory. This controller acts as a mediator that can route data through the high-bandwidth HBM path for frequent access or to the high-capacity NAND path for bulk storage, optimizing both bandwidth and capacity utilization.
2Quantity of substance
If non-volatile memories like NAND dies are used, then memory capacity is improved, but power requirements increase
Solution Approach 1:
The patent applies local quality by using different memory technologies for different access patterns. The HBM memory uses volatile technology optimized for low-power high-speed access to frequently used data, while the NAND memory uses non-volatile technology for power-efficient bulk storage of less frequently accessed data, reducing overall power consumption.
3Speed
If volatile memories like DRAM are used, then bandwidth is improved, but memory capacity is reduced
Solution Approach 1:
The patent merges volatile HBM memory and non-volatile NAND memory into an integrated package, combining the high bandwidth characteristics of volatile memory with the high capacity of non-volatile memory, allowing the system to achieve both high speed and large capacity simultaneously.
4Device complexity
If conventional memory configurations are used, then device complexity is reduced, but adaptability to specialized processing needs is reduced
Solution Approach 1:
The patent creates a universal memory system that can serve multiple specialized processing needs through a single integrated package. The combination of NAND and HBM memory with a configurable controller provides a multi-functional memory subsystem that can adapt to different processor types including GPUs, AI processors, and traditional CPUs, enhancing versatility without proportionally increasing complexity.
Data Source
AI summary
A processing core includes a multi-core processor integrated directly onto a high bandwidth, high-capacity non-volatile memory. The processor may for example be a large graphics processing unit (GPU) or artificial intelligence (AI) processor. The non-volatile memory may comprise a CBA (CMOS bonded to array) memory tile having a single large NAND memory tile coupled together with a CMOS logic circuit tile. The integrated processor and CBA memory tile may be affixed to an interposer. The processing core may further include stacks of high bandwidth memory (HBM) semiconductor dies affixed to the interposer around one or more sides of the processor and CBA memory tile.