RDMA Memory Region Hibernation for Tiered Storage Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for data management in distributed storage systems are inefficient and ineffective in handling data tiering without impacting performance or scalability, particularly due to the need for multiple round-trip times (RTTs) and resource-intensive client interactions.
Innovation Solution
A method and apparatus that utilize a server CPU and RDMA network interface card (RNIC) to determine suitability for hibernation, execute tier-down operations, and provide remote direct access to clients, allowing clients to manage tier-up or ad-hoc requests without additional RTTs, thereby optimizing data tiering handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If clients contact the distributed storage system to check data availability before each access, then data access accuracy is improved, but access time increases due to multiple round-trip times
Solution Approach 1:
The system performs preliminary actions by having the server proactively notify clients about data tier movements before clients need to access the data. This eliminates the need for clients to perform repeated availability checks, thereby reducing access time while maintaining accurate knowledge of data location.
Solution Approach 2:
The server implements a feedback mechanism where it actively informs clients about data location changes. Instead of clients repeatedly querying for data availability, the server pushes notifications about tier movements, creating an efficient feedback loop that reduces communication overhead and access time.
2Adaptability or versatility
If the server sends data from whichever tier it is available on, then data access flexibility is improved, but resource consumption increases due to CPU usage
Solution Approach 1:
Clients are empowered to directly access data from any tier based on server notifications about data location. The system enables self-service by providing clients with the information needed to autonomously retrieve data from the appropriate tier without requiring server-mediated data transfer operations, thereby reducing CPU resource consumption.
3Ease of operation
If all clients can request their data from the distributed storage system, then data accessibility is improved, but scalability deteriorates due to limited server resources
Solution Approach 1:
The system extracts the data location information from the server's internal state and communicates it to clients. This allows clients to independently determine where to access data without burdening the server with resource-intensive data retrieval operations, thereby improving scalability while maintaining ease of operation for all clients.
Data Source
AI summary
A method of data management in a data storage apparatus includes determining that a memory region (MR) stored in a first storage tier is suitable for hibernation. A tier-down operation is executed to move the MR to a second storage tier. In response to a remote direct memory access (RDMA) request to the MR from a client, a notification response is sent to the client to notify the client of the hibernated status of the MR. Moreover, in response to receiving a remote procedure call (RPC), from the client, the requested memory access is provided to the client. The requested memory access is provided to the client without any additional round-trip times (RTTs).