GPGPU Memory Segmentation for Capacity Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face limitations in memory capacity due to the difficulty in vertically stacking dynamic random-access memory (DRAM) dies and the inefficiency of using CPU memory as an extension, which restricts their ability to perform computationally intensive tasks in artificial intelligence and machine learning applications.
Innovation Solution
A processing system that includes external memory and GPGPUs, where data is moved from local memory to external memory when accessed infrequently and vice versa, optimizing memory access times to increase the effective capacity of the GPGPU memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in local memory to enable fast access, then access speed is improved, but memory capacity is limited
Solution Approach 1:
The memory system is segmented into multiple levels: fast local memory (HBM) for frequently accessed data and slower external memory (CPU memory) for less frequently accessed data. This segmentation allows the system to maintain high-speed access for critical data while expanding total capacity using external resources.
Solution Approach 2:
The patent transitions from a single-dimension memory architecture (only local HBM) to a multi-dimensional architecture that includes local memory, external memory, and cache layers. This dimensional expansion enables the system to overcome the capacity limitations of local memory while preserving fast access paths.
2Quantity of substance
If CPU memory is used as an extension of GPGPU memory, then memory capacity is increased, but access speed deteriorates
Solution Approach 1:
Different memory locations are assigned different access characteristics: frequently accessed data is kept in fast local HBM memory, while less frequently accessed data is stored in external CPU memory. This local quality differentiation ensures that the most performance-critical data paths maintain high speed while total capacity is expanded.
Solution Approach 2:
The cache memory acts as an intermediary between the GPGPU and external CPU memory. It buffers data transfers and manages the interface between the two memory systems, reducing the direct impact of slower external memory access on GPGPU performance.
3Quantity of substance
If more DRAM dies are vertically stacked to increase HBM capacity, then memory capacity is improved, but manufacturing difficulty increases
Solution Approach 1:
The patent makes CPU memory serve multiple functions: it acts as general-purpose system memory and simultaneously functions as an extended GPGPU memory resource. This multi-functionality eliminates the need for specialized high-capacity HBM configurations, reducing manufacturing complexity while providing expanded memory capacity.
Data Source
AI summary
The maximum capacity of a very fast memory in a system that requires very fast memory access times is increased by adding a memory with remote access times that are slower than required, and then moving infrequently accessed data from the memory with the very fast access times to the memory with the slow access times.