RDMA Storage Access Control via Curator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face high and unpredictable latency due to mismatched processor and storage resource utilization, where processor-heavy utilization leaves storage requests unserviced despite available storage bandwidth.
Innovation Solution
Implementing a distributed storage system with memory hosts that use remote direct memory access (RDMA) and access control lists, allowing direct client access to storage resources without processor intervention, and managing connections through a curator for efficient memory allocation and access control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If processor and storage resources are coupled in a distributed storage system, then storage requests can be serviced through software processing, but latency becomes high and unpredictable when processors are heavily utilized
Solution Approach 1:
The patent extracts the storage request servicing function from the processor by implementing direct memory access hardware that allows storage requests to be handled independently of processor availability. The storage system can service requests directly through hardware-level operations, removing the bottleneck caused by processor utilization and thereby reducing latency while maintaining reliable storage access.
2Productivity
If dedicated processors are used to service storage requests, then storage requests can be serviced immediately even when other processors are busy, but processing capacity for nominal processing tasks decreases
Solution Approach 1:
The patent segments the storage system into independent storage resources that can operate autonomously without requiring dedicated processors. By dividing the storage functionality into self-sufficient units with direct memory access capabilities, the system achieves high storage throughput without allocating separate processors, thereby avoiding the trade-off between storage productivity and overall system complexity.
3Loss of time
If remote direct memory access is implemented without access control, then direct client access to storage resources is enabled with low latency, but security and access control requirements cannot be met
Solution Approach 1:
The patent implements preliminary access control by establishing access control lists before direct memory access operations occur. The system pre-configures permission structures and validates client access rights in advance, allowing fast direct memory access to proceed only for authorized clients. This preliminary action ensures both low latency for legitimate accesses and proper security control.
Solution Approach 2:
The patent introduces an access control list as an intermediary structure that mediates between direct memory access requests and security requirements. The access control list acts as a reference mechanism that enables fast validation of client permissions without blocking the direct access path, thereby maintaining low latency while ensuring proper access control through the intermediary lookup process.
Data Source
AI summary
A distributed storage system that includes memory hosts having non-transitory memory storing data chunks of files. Each data chunk is associated with an access control list. Each memory host also includes a network interface controller in communication with the memory for servicing remote direct memory access requests and a computing processor in communication with the memory and the network interface controller. The computing processor executes a host process that registers a set of remote direct memory accessible regions of the memory storing data chunks with the network interface controller. In response to receiving a connection request from a client process of a client to access a data chunk, the host process establishes a remote direct memory access capable connection with the client process when both the client and the data chunk are associated with the same access control list.