GPU–SSD Key-Value Data Path for Asynchronous ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Related art SSDs with a block interface require CPU involvement for key value access during stochastic machine learning training, leading to serialized access and limited GPU utilization due to PCIe bus traversal and file system access.
Innovation Solution
A system with an onboard SSD and GPU direct communication via peer-to-peer DMA, allowing asynchronous key value access through a key value request queue and memory-mapped I/O, enabling concurrent GPU operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU is involved in key value access during machine learning training, then data access can be performed through standard file system interfaces, but GPU utilization is limited due to serialized access and PCIe bus traversal
Solution Approach 1:
The patent extracts the key value access functionality from the CPU file system and implements it directly in the SSD device. The SSD now contains a key value store that can directly respond to key value requests without CPU intervention, separating this specific function from the general-purpose file system while maintaining ease of access through dedicated interfaces.
Solution Approach 2:
The patent introduces a dedicated key value interface as an intermediary between the GPU and SSD, bypassing the CPU and traditional file system. This intermediary layer enables direct key value requests from GPU to SSD, eliminating the serialization bottleneck while maintaining simple access patterns for the GPU.
2Ease of operation
If CPU performs file index lookup and file system access to identify data location, then data can be accessed through standardized interfaces, but access time increases due to serialized operations
Solution Approach 1:
The patent implements preliminary action by pre-organizing data in the SSD into a key value store structure during system initialization or data loading phases. This pre-processed structure enables direct key-based lookup without requiring runtime file index traversal or file system operations, significantly reducing access time while maintaining ease of data retrieval.
3Adaptability or versatility
If key value access requires PCIe bus traversal and CPU involvement, then system compatibility is maintained, but performance is limited by bus bandwidth and processing overhead
Solution Approach 1:
The patent segments the data access path into two distinct channels: a control channel for metadata and key value requests, and a data channel for actual training data transfer. This segmentation allows key value operations to proceed independently through a dedicated interface while data transfer utilizes the full PCIe bandwidth, improving both compatibility and performance.
Data Source
AI summary
A system and method for machine learning. The system includes a GPU with a GPU memory, and a key value storage device connected to the GPU memory. The method includes, writing, by the GPU, a key value request to a key value request queue in a input-output region of the GPU memory, the key value request including a key. The method further includes reading, by the key value storage device, the key value request from the key value request queue, and writing, by the key value storage device, in response to the key value request, a value to the input-output region of the GPU memory, the value corresponding to the key of the key value request.