GPU Memory Access API for Host Storage Allocation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Processors, particularly graphics processing units (GPUs), often require more memory than what is available on a single GPU, lacking the ability to allocate memory outside of the GPU, leading to inefficient resource usage.

Innovation Solution

Implementing an application programming interface (API) that allows processors to indicate whether to access GPU storage or host storage, enabling memory allocation based on flags in memory, thereby allowing memory to be allocated on either GPU or host devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If a single GPU is used for processing, then the processing capability is sufficient, but the memory capacity is insufficient for programs that need more storage

Engineering Contradiction:

Improvememory capacityVSAvoidmemory allocation flexibility

Core Design Contradiction:

Quantity of substanceVSAdaptability or versatility

Solution Approach 1:

The patent combines GPU memory and host memory into a unified memory space, allowing programs to allocate memory from either source seamlessly. The memory allocation system merges two previously separate memory pools into one accessible resource, resolving the contradiction between limited GPU memory capacity and the need for greater storage without sacrificing allocation flexibility.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The unified memory allocation system allows the same memory pool to serve multiple functions - acting as both GPU memory and host memory depending on allocation needs. This multi-functional approach enables the system to adapt to different memory capacity requirements while maintaining a single, flexible allocation interface.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Speed

If memory is allocated only on GPU, then access speed is fast, but total available memory is limited

Engineering Contradiction:

Improvememory access speedVSAvoidtotal memory available

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The system applies local quality by allowing different regions of the unified memory space to have different characteristics - some portions optimized for fast GPU access, others for larger host memory capacity. The allocation system can direct specific memory allocations to GPU memory for speed-critical operations or to host memory for capacity requirements, maintaining both speed and quantity.

Inventive Principle:

Principle #3Local quality

3Adaptability or versatility

If unified memory allocation is implemented, then memory capacity and flexibility are improved, but control over physical memory location is reduced

Engineering Contradiction:

Improvememory allocation flexibilityVSAvoidcontrol over memory location

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The unified memory system implements dynamics by making the memory allocation behavior adaptive rather than static. The system can dynamically adjust allocation strategies based on runtime conditions, program requirements, and resource availability. This dynamic approach maintains flexibility while providing programmatic control mechanisms for guiding memory location decisions when needed.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20250284410A1Application programming interface to indicate memory access

Publication Date: 2025.09.11 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to perform a neural network to perform an API to cause storage to be reserved. In at least one embodiment, for example, an API causes storage to be reserved based, at least in part, on a flag indicating a memory pool to be allocated to a memory of a processor, such as a GPU or CPU. In at least one embodiment, as another example, a processor comprising one or more circuits performs an application programming interface (API) to indicate whether one or more graphics processing units (GPU) are to access GPU storage or host storage.