Partitioned Key-Value Store Atomic Memory Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing partitioned key-value stores face challenges in efficiently handling large amounts of data due to the need for parallelism and the complexity of key-value data transfer and processing, particularly in high-performance computing environments, where server-side atomic memory operations are lacking.
Innovation Solution
Implementing server-side atomic memory operations in a partitioned key-value store using the Multidimensional Data Hashing Indexing Middleware (MDHIM) framework, which enables Remote Direct Memory Access (RDMA) operations and offloads Input/Output (I/O) computations to the server, allowing clients to perform other tasks while the server processes data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If server performs atomic memory operations by reading from and writing to client-side memory locations, then data transfer efficiency is improved, but network communication overhead increases
Solution Approach 1:
The server acts as an intermediary that performs atomic memory operations by reading from and writing to client-side memory locations. This mediator approach allows the server to manage data transfer efficiently while the client maintains ownership of its memory space, balancing performance improvement with controlled network communication.
Solution Approach 2:
The patent replaces traditional mechanical I/O operations with server-side atomic memory operations that directly access client memory locations. This substitution eliminates the need for traditional client-side I/O processing, improving data transfer efficiency by performing operations directly in memory without intermediate mechanical processing steps.
2Ease of operation
If server offloads I/O computations to perform server-side processing, then client computational workload is reduced, but server processing complexity increases
Solution Approach 1:
The server performs self-service by autonomously executing atomic memory operations and managing its own I/O computations. This allows the server to handle its own processing needs without requiring extensive client intervention, reducing client workload while the server manages its own complexity through standardized atomic operations.
Solution Approach 2:
The patent segments computational tasks by separating I/O operations from computational processing. The server handles I/O computations independently through atomic memory operations, while clients focus purely on computational workloads. This segmentation reduces client complexity while distributing server processing complexity across multiple independent atomic operations.
3Quantity of substance
If partitioned key-value store uses parallel nodes for storing large data, then data storage capacity is improved, but system coordination complexity increases
Solution Approach 1:
The key-value store is segmented across multiple parallel nodes, with each node responsible for a specific partition of data. This segmentation enables scalable data storage capacity while reducing system coordination complexity by localizing operations to specific nodes rather than requiring global coordination across all nodes.
Solution Approach 2:
Each parallel node maintains local quality by independently managing its own data partition with consistent data structures and operations. This local quality approach allows each node to operate autonomously with standard atomic memory operations, improving data storage capacity without proportionally increasing system coordination complexity.
Data Source
AI summary
A partitioned key-value store is provided that supports atomic memory operations. A server performs a memory operation in a partitioned key-value store by receiving a request from an application for at least one atomic memory operation, the atomic memory operation comprising a memory address identifier; and, in response to the atomic memory operation, performing one or more of (i) reading a client-side memory location identified by the memory address identifier and storing one or more key-value pairs from the client-side memory location in a local key-value store of the server; and (ii) obtaining one or more key-value pairs from the local key-value store of the server and writing the obtained one or more key-value pairs into the client-side memory location identified by the memory address identifier. The server can perform functions obtained from a client-side memory location and return a result to the client using one or more of the atomic memory operations.