SmartNIC Persistent Caching for Low-Latency Block I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud storage systems experience high latency and jitter due to multiple network hops required for block input/output operations, which affects user experience and efficiency.
Innovation Solution
A smart network interface card (smartNIC) manages persistent storage locally, utilizing a caching layer to reduce latency by processing NVMe protocol data directly and caching read operations, while write operations are committed to persistent memory before being synced to remote storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transported through multiple network hops including smartNIC and NVMe target, then remote block storage accessibility is achieved, but end-to-end latency increases
Solution Approach 1:
The smartNIC maintains a local persistent storage cache that pre-loads frequently accessed data blocks before they are actually needed by host instances. This preliminary caching action eliminates the need for real-time network round trips for common read operations, significantly reducing end-to-end latency while maintaining remote storage accessibility.
Solution Approach 2:
The persistent storage cache at the smartNIC acts as an intermediary layer between the host instances and the remote NVMe target. It mediates data requests by serving frequently accessed data locally while forwarding less frequently accessed data to the remote storage, thus decoupling the latency-sensitive read operations from the network path.
2Extent of automation
If multiple network hops are used for block I/O operations, then centralized storage management is achieved, but packet drops and delays increase
Solution Approach 1:
The storage access path is segmented into two independent paths: a local persistent storage path for frequently accessed data and a remote NVMe path for less frequently accessed data. This segmentation allows the system to optimize each path independently, using local storage for high-reliability operations and remote storage for centralized management, thereby improving overall packet delivery reliability.
3Loss of time
If persistent storage is cached at smartNIC, then read operation latency is reduced, but write operation complexity increases
Solution Approach 1:
For write operations, the smartNIC performs preliminary local caching of data blocks before they are committed to the remote storage. The write path includes: (1) writing to local persistent storage for immediate acknowledgment, (2) asynchronously flushing to remote NVMe storage, and (3) maintaining a consistency log to track pending writes. This preliminary local write action reduces read latency while managing write complexity through structured async processing.
Solution Approach 2:
The smartNIC implements feedback mechanisms to track write status between local cache and remote storage. When a write operation is initiated, the system monitors the propagation status and provides feedback to ensure consistency. This feedback loop manages the complexity of coordinated writes by automatically handling edge cases and ensuring data integrity across both storage layers.
Data Source
AI summary
The disclosed systems, methods, and computer readable media relate to managing persistent memory within a smart network interface card (smartNIC) of a cloud-computing environment, or at storage device of the host instance on which the smartNIC operates and to which the smartNIC has access. The smartNIC may be configured with various modes that enable passthrough of input/output operations to or from non-volatile remote storage (e.g., a block storage data plane volume) or storage of at least some of the data corresponding to the input/output operations within the persistent memory managed by the smartNIC. The smartNIC may be configured to select a storage location for data between a pool of candidate storage including the persistent storage managed by the smartNIC or the non-volatile remote storage. These techniques improve communications between the smartNIC and the non-volatile remote storage and reduce the risk of network jitters and overall network latency.