Shared GPU Memory Access via DMA Eliminating Data Copying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-GPU systems require unnecessary data copying steps when combining output data from multiple GPUs, as each GPU can only access and write to its own pinned memory buffer, limiting efficiency and performance in workload subdivision.
Innovation Solution
A method is introduced where a GPU can directly access and write to shared memory buffers allocated by other GPUs, eliminating the need for data copying through Direct Memory Access (DMA), by maintaining lists of processes and address ranges, and mapping these ranges into the GPU's address space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each GPU is allocated its own pinned memory buffer, then each GPU can independently access its allocated memory through DMA, but data copying is required to combine outputs from multiple GPUs into a single buffer
Solution Approach 1:
The pinned memory buffer is designed to be universally accessible by multiple GPU contexts simultaneously. Instead of allocating exclusive memory buffers to each GPU, the system creates a shared pinned memory buffer that can be accessed by multiple GPUs through DMA, eliminating the need for data copying while maintaining reliable memory access for all GPUs.
Solution Approach 2:
The patent merges the memory access capabilities of multiple GPUs into a single shared pinned memory buffer. By combining the output destinations of multiple GPUs into one shared buffer, the system eliminates the copying step required when each GPU had its own separate buffer, thereby improving data consolidation efficiency.
2Device complexity
If pinned memory buffers are exclusively allocated to individual GPU contexts, then memory management is simplified, but application developers must implement manual copying to consolidate results from multiple GPUs
Solution Approach 1:
The pinned memory buffer is designed with universal accessibility, allowing multiple GPU contexts to access the same buffer simultaneously. This multi-functional design enables the buffer to serve as both an input and output destination for multiple GPUs, eliminating the need for manual copying operations while maintaining manageable memory allocation through the driver's address range tracking.
3Reliability
If multiple GPUs write to separate pinned memory buffers, then each GPU has dedicated access, but the application requires additional copying steps to gather results into a single buffer
Solution Approach 1:
The patent merges the destination buffers of multiple GPUs into a single shared pinned memory buffer. By combining the memory resources, the system eliminates the time-consuming copying operation required to gather results from multiple separate buffers, while maintaining reliable GPU access through the shared buffer's pinned memory characteristics.
Solution Approach 2:
The shared pinned memory buffer enables continuous data flow from multiple GPUs without interruption for copying operations. Multiple GPUs can write their results directly to the shared buffer simultaneously, maintaining continuous useful action and eliminating the idle copying time that would occur with separate buffers.
Data Source
AI summary
One or more embodiments of the invention set forth techniques to create a process in a graphical processing unit (GPU) that has access to memory buffers in the system memory of a computer system that are shared among a plurality of GPUs in the computer system. The GPU of the process is able to engage in Direct Memory Access (DMA) with any of the shared memory buffers thereby eliminating additional copying steps that have been needed to combine data output of the various GPUs without such shared access.