NVRAM Shared Storage with RDMA for Database Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The shared storage architecture in database management systems (DBMS) faces challenges in accessing database data efficiently due to dependence on common network connections and processing capacity, leading to increased latency and overhead in read and write operations.
Innovation Solution
Implementing a non-volatile random access memory (NVRAM) based shared storage architecture that allows direct data transfer between NVRAMs using remote direct memory access (RDMA), enabling one-sided read and write operations with reduced latency and processor overhead, and using storage services to manage data block locations and mappings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional shared storage architecture is used with block storage devices, then data persistence is achieved, but access latency and processor overhead increase
Solution Approach 1:
The patent changes the storage medium parameter from traditional block storage devices to NVRAM, fundamentally altering the speed and persistence characteristics. NVRAM provides persistent storage with memory-speed access, resolving the contradiction between data persistence and access latency by changing the physical state and material properties of the storage medium.
Solution Approach 2:
The patent replaces mechanical block storage systems with electronic NVRAM-based storage. By substituting the mechanical seek and read operations with direct electronic memory access, the system eliminates the inherent latency of mechanical storage while maintaining data persistence through NVRAM's non-volatile properties.
2Reliability
If traditional shared storage architecture is used with block storage devices, then data persistence is achieved, but processor overhead and network dependence increase
Solution Approach 1:
The patent implements self-service through storage services that automatically manage data block locations, mappings, and transfers between NVRAMs. This automation eliminates the need for complex processor-mediated storage management, reducing overhead while maintaining persistent storage capabilities through service-level automation rather than processor-level intervention.
Solution Approach 2:
The patent introduces storage services as intermediaries between computing nodes and NVRAM storage. These services handle the complexity of data management, location tracking, and block transfers, shielding processors from complex storage management tasks and reducing their overhead while ensuring data persistence through service-level protocols.
3Productivity
If NVRAM-based shared storage with RDMA is implemented, then access speed and availability improve, but data integrity management becomes more challenging
Solution Approach 1:
The patent implements feedback mechanisms through storage services that track data block locations, monitor transfer status, and verify data integrity. The system continuously updates mapping information and confirms successful writes between NVRAMs, providing feedback loops that ensure data integrity despite the high-speed direct memory access that bypasses traditional processor checks.
Solution Approach 2:
The patent performs preliminary actions by pre-establishing data block mappings and location information before actual data transfers occur. Storage services prepare and validate data block metadata, locations, and transfer parameters in advance, ensuring data integrity is managed proactively rather than reactively, even as data moves at high speed between NVRAMs.
Data Source
AI summary
A shared storage architecture persistently stores database files in non-volatile random access memories (NVRAMs) of computing nodes of a multi-node DBMS. The computing nodes of the multi-node DBMS not only collectively store database data on NVRAMs of the computing nodes, but also host database server instances that process queries in parallel, host database sessions and database processes, and together manage access to a database stored on the NVRAMs of the computing nodes. To perform a data block read operation from persistent storage, a data block may be transferred directly over a network between NVRAM of a computing node that persistently stores the data block to a database buffer in non-volatile RAM of another computing node that requests the data block. The transfer is accomplished using remote direct memory access (“RDMA).