Direct Memory Access for GPU Memory via Peer-to-Peer Bus
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face inefficiencies in data transfer between peripheral devices and graphics processing unit (GPU) memory, as existing technologies require data to be copied through central processing unit (CPU) memory, slowing down data processing and limiting performance in applications like artificial intelligence and machine learning.
Innovation Solution
The implementation of direct memory access (DMA) techniques that allow data to be transferred directly between peripheral devices and GPU memory without going through CPU memory, using data structures like struct page data structures to map bus addresses to physical GPU memory, enabling direct input/output operations and reducing the need for CPU intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred through CPU memory between storage device and GPU memory, then data transfer can be performed using existing memory management structures, but data transfer speed is reduced and processing time increases
Solution Approach 1:
The patent extracts the CPU memory intermediary from the data transfer path between storage device and GPU memory. By creating direct mapping between storage device and GPU memory through DMA, the CPU memory is taken out of the critical data transfer path, eliminating the copy operations that slow down processing while maintaining compatibility with existing memory management structures.
Solution Approach 2:
The patent introduces a DMA buffer and mapping mechanism as an intermediary that enables direct communication between storage device and GPU memory. This intermediary allows the storage device to write directly to GPU memory addresses without CPU involvement, while still using standard memory management data structures for address translation and memory region tracking.
2Productivity
If direct memory access is implemented for GPU memory, then data transfer speed increases, but system complexity increases due to additional mapping and address translation requirements
Solution Approach 1:
The patent makes existing memory management data structures universal by extending them to handle both CPU memory and GPU memory regions. The same data structures that track CPU virtual and physical memory are used to track GPU memory regions, enabling a unified approach to memory management that reduces complexity despite the added functionality of GPU-direct access.
Solution Approach 2:
The system implements self-service memory management where the DMA controller automatically performs address translation and memory mapping using the existing data structures. The CPU does not need to manually manage the complexity of GPU memory mapping, as the system uses the same memory management mechanisms that already exist for CPU memory to handle GPU address translation automatically.
3Ease of operation
If data is copied through CPU memory, then memory address translation and management is simplified, but data transfer efficiency decreases
Solution Approach 1:
The patent segments memory management into distinct regions (CPU virtual memory, CPU physical memory, GPU virtual memory, GPU physical memory) while maintaining a unified data structure approach. Each memory region is tracked separately with appropriate mapping relationships, allowing efficient direct access paths while preserving the simplicity of standardized memory management operations for each segment.
Data Source
AI summary
Direct memory access (DMA) is provided in a computing system that includes a central processing unit (CPU), CPU memory associated with the CPU, a graphics processing unit (GPU), GPU memory associated with the GPU, a storage device capable of direct memory access, and a peer-to-peer host bus to which the other components are electrically coupled, directly or indirectly. For each page of the GPU physical memory, a data structure representing the page of GPU physical memory is generated, a GPU virtual memory space is allocated, the GPU virtual memory space is mapped to a GPU physical memory space. Based on the data structure representing the page of GPU physical memory, the GPU physical memory space is mapped to a CPU virtual address associated with a user-space process. A direct input/output operation on the storage device is initiated using the CPU virtual address, which is mapped to the GPU physical memory, at least a bus address is generated based on the data structure representing the page of GPU physical memory, and a DMA operation is initiated based on the bus address to transfer data between the storage device and the GPU physical memory space through the peer-to-peer host bus without copying the data to the CPU memory.