Local Storage Engine With SSD Direct Transfer Bypassing System Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems suffer from increased latency and resource consumption due to multiple folds of memory copy operations involving the CPU and system memory during data transfers within and between storage servers, leading to inefficiencies in system performance and resource utilization.
Innovation Solution
A system that bypasses system memory by allocating a region of volatile memory of a first storage drive to be directly accessible by a second storage drive or network interface card, allowing data transfer without involving the system memory, and utilizing a local storage engine to manage metadata and write data to non-volatile memory with separate control and data paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data transfers use the traditional path through system memory and CPU, then data can be transferred between storage drives, but latency increases and system memory bandwidth is consumed
Solution Approach 1:
The patent segments the data transfer path by creating a dedicated direct memory access channel between storage drives that bypasses the system memory and CPU. This is achieved by allocating a portion of system memory as a buffer zone that storage drives can access directly, dividing the transfer process into independent segments that operate in parallel without contending for system resources.
Solution Approach 2:
The patent introduces an intermediary buffer memory region that acts as a mediator between storage drives. This buffer allows drives to exchange data directly without requiring system memory as an intermediate storage location, eliminating the need for data to be copied through the system memory while still providing a controlled interface for data transfer.
2Reliability
If data transfers involve multiple memory copy operations through system memory, then data can be moved between storage drives, but memory bandwidth consumption increases
Solution Approach 1:
The patent extracts the data transfer function from the system memory path and creates a separate dedicated transfer channel. By taking out the data movement operation from the general-purpose system memory pathway, the invention eliminates unnecessary memory access operations and reduces the bandwidth burden on system memory while maintaining full data transfer capability.
3Reliability
If data transfers use the traditional path through CPU and system memory, then data can be transferred between storage drives, but CPU utilization increases
Solution Approach 1:
The patent enables storage drives to perform self-service data transfers by directly accessing each other through the allocated buffer memory region. This eliminates the need for CPU intervention in the data transfer process, allowing drives to autonomously manage their own data exchange operations and freeing the CPU for more productive computational tasks.
Data Source
AI summary
One embodiment provides a system which facilitates data movement. The system allocates, in a volatile memory of a first storage drive, a first region to be accessed directly by a second storage drive or a first NIC. The first storage drive, the second storage drive, and the first NIC are associated with a first server. The system stores data in the first region. Responsive to receiving a first request from the second storage drive to read the data, the system transmits, by the first storage drive to the second storage drive, the data stored in the first region while bypassing a system memory of the first server. Responsive to receiving, from a third storage drive associated with a second server, a second request to read the data, the system retrieves, by the first NIC, the data stored in the first region while bypassing the system memory of the first server.