CPU Mailbox Read Access to GPU Memory for Efficient Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data transfer protocols in computing environments, particularly for large data blocks between CPUs and GPUs, face inefficiencies due to limited peer-to-peer memory buffers, leading to low data transfer rates and resource competition among GPUs, with DMA transfers being cumbersome and inappropriate for urgent memory transfers.
Innovation Solution
A method involving multiple read commands for transferring data blocks, where each command accesses a portion of the data and stores it in different parts of a buffer memory, allowing simultaneous operations without consuming additional peer-to-peer memory buffers, and using a circular read buffer to manage data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple peer-to-peer memory buffers are used to transfer large data blocks simultaneously, then data transfer rate is improved, but the number of available buffers is limited and other peer GPUs cannot access buffers when one GPU uses multiple buffers
Solution Approach 1:
The patent segments a large data block transfer into multiple smaller sub-block transfers, each fitting within a single peer-to-peer memory buffer. The source GPU divides the large data block into sub-blocks and transfers them sequentially through the mailbox protocol, allowing the buffer to be reused for each sub-block while maintaining efficient data flow.
Solution Approach 2:
The patent introduces a new dimension of operation by allowing the source GPU to initiate multiple independent read commands to different target GPUs simultaneously. This transforms the traditional single-buffer sequential model into a multi-command parallel model where the limiting factor shifts from buffer quantity to command processing capacity.
2Adaptability or versatility
If a single peer-to-peer memory buffer is used for large data block transfers, then buffer availability for other GPUs is maintained, but data transfer rate decreases due to funneling through a small data path
Solution Approach 1:
The patent implements continuous data transfer by allowing the source GPU to pipeline multiple read commands through the mailbox protocol. While one transfer is in progress, the source GPU can prepare and issue subsequent commands, ensuring the data path remains continuously utilized without idle buffer waiting time.
Solution Approach 2:
The source GPU prepares and issues multiple read commands in advance before all data transfers are complete. This preliminary action allows the mailbox protocol to process commands in parallel with data movement, reducing overall transfer latency and improving throughput.
3Productivity
If DMA operation is used to transfer large data blocks, then data transfer rate is improved, but latency is long and initiation is cumbersome making it inappropriate for urgent transfers
Solution Approach 1:
The patent introduces the mailbox protocol as an intermediary mechanism between direct peer-to-peer access and DMA transfers. The mailbox acts as a lightweight control structure that enables rapid command issuance and completion notification, providing low-latency transfers for urgent data while maintaining high throughput through pipelined operations.
Data Source
AI summary
Techniques are disclosed for peer-to-peer data transfers where a source device receives a request to read data words from a target device. The source device creates a first and second read command for reading a first portion and a second portion of a plurality of data words from the target device, respectively. The source device transmits the first read command to the target device, and, before a first read operation associated with the first read command is complete, transmits the second read command to the target device. The first and second portions of the plurality of data words are stored in a first and second portion a buffer memory, respectively. Advantageously, an arbitrary number of multiple read operations may be in progress at a given time without using multiple peer-to-peer memory buffers. Performance for large data block transfers is improved without consuming peer-to-peer memory buffers needed by other peer GPUs.