Integrated GPU Memory Core With CBA NAND and HBM Bandwidth

The patent transitions from traditional planar memory arrangements to three-dimensional stacked memory architectures (HBM stacks with multiple DRAM dies stacked vertically). This vertical stacking enables significantly higher bandwidth rates by providing multiple simultaneous data access paths while maintaining high memory capacity, thereby resolving the contradiction between capacity and bandwidth.

The patent integrates multiple memory types (volatile DRAM and non-volatile NAND) into a unified stacked memory system with the processor. This hybrid architecture combines the high bandwidth of volatile memory with the high capacity of non-volatile memory, allowing the system to achieve both high capacity and high bandwidth simultaneously through coordinated operation of different memory types.

The patent assigns different memory types to different functional requirements within the same memory system. Volatile DRAM stacks provide high-speed caching for frequently accessed data with low power consumption during active use, while non-volatile NAND stacks provide bulk storage capacity. This local differentiation of memory characteristics allows the system to achieve high capacity without uniformly high power requirements across all memory operations.

The system employs periodic data transfer between the volatile and non-volatile memory stacks, moving data only when necessary. This reduces overall power consumption by keeping data in low-power non-volatile storage and only activating the higher-power volatile memory and data transfer mechanisms when actual data access is required, thereby maintaining high capacity with reduced average power requirements.

The patent implements a nested memory hierarchy where volatile DRAM stacks are positioned closer to the processor for high-speed access to frequently used data, while non-volatile NAND stacks provide the outer layer of bulk storage. This nested arrangement allows the system to achieve high bandwidth for active operations through the inner volatile layer while maintaining large total capacity through the outer non-volatile layer.

The memory system is segmented into multiple independent stacks with different characteristics - volatile DRAM stacks optimized for bandwidth and non-volatile NAND stacks optimized for capacity. This segmentation allows each stack type to be optimized for its specific function while working together as a unified memory system, achieving both high bandwidth and high capacity through coordinated operation of the segmented components.