Direct Storage-to-GPU Data Transfer via NVLink Interfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for transferring data between storage devices and processing elements, such as GPUs, suffer from latency issues due to the use of host CPU memory as an intermediate hop and performance limitations in peer-to-peer solutions like PCIe.
Innovation Solution
A storage device with a network interface that allows direct data transfer to processing elements over a data communications network, bypassing the host CPU, using interfaces like NVLink to facilitate point-to-point connections and expose its memory for direct access by processing elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is transferred through host CPU memory as an intermediate hop, then compatibility and ease of operation are improved, but latency and transfer speed worsen
Solution Approach 1:
The patent introduces a storage device with direct network interface (NVLink) as an intermediary that enables peer-to-peer data transfer between GPU and storage, bypassing the host CPU memory intermediate hop while maintaining system compatibility through standardized interfaces
Solution Approach 2:
The patent segments the data transfer path into independent components: storage device with network interface, GPU with direct network access, and host CPU for control only. This segmentation allows parallel operation and eliminates the sequential bottleneck of traditional CPU-mediated transfer
2Speed
If peer-to-peer solutions like PCIe are used, then transfer speed is improved, but device complexity and compatibility requirements worsen
Solution Approach 1:
The storage device is designed with dual interface capability: traditional host interface for CPU communication and NVLink interface for direct GPU communication. This multi-functionality allows the same device to serve both legacy and high-performance needs without increasing overall system complexity
Solution Approach 2:
The NVLink interface acts as a specialized intermediary that simplifies the peer-to-peer connection between storage and GPU, handling protocol translation and data routing while presenting a standardized interface to both endpoints
3Ease of operation
If host CPU memory is used as intermediate storage, then ease of operation and compatibility are improved, but data transfer efficiency and productivity worsen
Solution Approach 1:
The patent introduces a storage device with direct network interface (NVLink) as an intermediary that enables peer-to-peer data transfer between GPU and storage, bypassing the host CPU memory intermediate hop while maintaining system compatibility through standardized interfaces
Solution Approach 2:
The storage device performs self-service by directly managing data transfer to the GPU through its own network interface, without requiring host CPU intervention for the actual data movement, thereby improving productivity while maintaining ease of operation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for moving data between a storage device and a processing element are disclosed. The storage device may include: a non-volatile storage medium; a first interface for communicating with a computing device; and a second interface for communicating with a processing element over a data communications network. The processing element may include a processing circuit that is configured to: receive a first request from the computing device via the first interface, where the first request identifies a first memory address associated with the first memory; retrieve first data from the non-volatile storage medium based on the first request; and transmit the first data to the processing element via the second interface for storing the first data in the first memory based on the first memory address.