Direct NVM Access via Virtual Storage Driver for GPU Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data transfer processes between non-volatile memory (NVM) and local memory for graphics processing units (GPUs) involve a two-hop process through system memory, leading to increased traffic and congestion, as the GPU must first access data from system memory after it is transferred from NVM.
Innovation Solution
A method and system that directly access and transfer data between NVM and local memory by treating them as part of physical memory, using a virtual storage driver to redirect data access, allowing the GPU to access NVM as if it were regular low-latency memory, thereby simplifying data access and reducing intermediate steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data transfer uses the traditional two-hop process through system memory, then data can be transferred between NVM and local memory, but traffic and congestion increase
Solution Approach 1:
The patent introduces a memory management unit (MMU) and translation lookaside buffer (TLB) as intermediary components that enable direct address translation between GPU virtual addresses and NVM physical addresses. This mediator allows the GPU to access NVM directly without requiring data to hop through system memory, thus eliminating traffic congestion while maintaining data transfer functionality
Solution Approach 2:
The patent segments the memory address translation function into multiple components: a page table structure stored in system memory, an MMU for address translation, and a TLB for caching recent translations. This segmentation allows the system to handle large NVM address spaces efficiently while enabling direct GPU access to NVM, avoiding the need for data to traverse through system memory
2Productivity
If data is transferred from NVM to system memory first, then the GPU can access the data, but latency increases
Solution Approach 1:
The MMU and TLB act as intermediaries that enable direct address translation and access paths from GPU to NVM. The TLB specifically caches recent address translations to minimize the time required for address resolution, allowing the GPU to access NVM data in near-real-time without the latency penalty of transferring data through system memory first
Solution Approach 2:
The system performs preliminary address translation and sets up direct access paths before the GPU needs to access NVM data. The page table structures are pre-configured and the TLB pre-loads recent translations, so when the GPU initiates a data access operation, it can proceed directly to NVM without waiting for data to be transferred through system memory
3Ease of operation
If the GPU accesses NVM as block addressable memory, then data can be transferred, but the access process becomes complex
Solution Approach 1:
The patent makes the NVM accessible to the GPU through a universal address translation mechanism that works with standard GPU memory access instructions. The MMU and TLB provide a unified interface that translates GPU virtual addresses to NVM physical addresses, allowing the GPU to access NVM using the same byte-addressable methods it uses for local memory, thereby simplifying the access process despite the underlying complexity of NVM's block-addressable nature
Data Source
AI summary
Described herein is a method and system for directly accessing and transferring data between a first memory architecture and a second memory architecture associated with a graphics processing unit (GPU) by treating the first memory architecture, the second memory architecture and system memory as a single physical memory, where the first memory architecture is a non-volatile memory (NVM) and the second memory architecture is a local memory. Upon accessing a virtual address (VA) range by a processor, the requested content is paged in from the single physical memory and is then redirected by a virtual storage driver to the second memory architecture or the system memory, depending on which of the GPU or CPU triggered the access request. The memory transfer occurs without awareness of the application and the operating system.