RDMA Key-Value Store Collision Remediation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing key-value store designs, such as Nessie, face issues with brownout failures, insertion failures, and lookup failures during collision remediation and concurrent operations, limiting the availability and performance of RDMA-enabled key-value stores when multiple processes access the store simultaneously.
Innovation Solution
The implementation of a RDMA-enabled key-value store (RKVS) allows for lock-free multi-process operations through the use of header flags in hash table entries to manage collisions and concurrent operations, enabling continued access and operation during collision remediation, and preventing insertion and lookup failures by utilizing flags such as Allocd, Influx, Time, and Fence flags to manage key-value pair states and priorities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple processes perform RDMA read and write operations directly in the key-value store, then throughput and scalability are improved, but insertion failures and lookup failures occur during concurrent operations
Solution Approach 1:
The patent implements dynamic state tracking using header flags (Allocd, Influx, Time, Fence) that change based on the current operation state. These flags dynamically indicate whether an entry is being allocated, is in an influx operation, is undergoing collision remediation, or has a fence set, allowing concurrent processes to adapt their behavior based on real-time store state
Solution Approach 2:
The patent uses header flags as feedback mechanisms that provide information about the state of hash table entries to concurrent processes. When a process reads an entry's headers, it receives feedback about the entry's current state (e.g., Influx flag set, Time flag set), which guides the process's subsequent actions to avoid conflicts and failures
2Reliability
If collision remediation is performed by stopping all read and write operations, then collision resolution is achieved, but availability and performance deteriorate during remediation
Solution Approach 1:
The patent sets the Time flag in entry headers as a preliminary action to mark entries that are undergoing or will undergo collision remediation. This preliminary marking allows other processes to identify and avoid these entries without requiring a complete system stop, enabling continued operation of non-conflicting processes
Solution Approach 2:
The patent segments the collision remediation process by individually marking only the specific entries involved in collisions with the Time flag, rather than stopping all operations in the entire key-value store. This allows collision remediation to proceed for affected entries while other entries remain accessible to concurrent processes
3Productivity
If lock-free multi-process operations are implemented, then scalability and performance are improved, but insertion failures occur during concurrent write operations
Solution Approach 1:
The patent uses header flags as intermediary mechanisms that mediate between concurrent write operations. The Allocd, Influx, and Fence flags act as intermediaries to coordinate access to hash table entries, allowing lock-free operations while preventing insertion failures through structured conflict resolution
Solution Approach 2:
The patent implements self-service collision resolution where processes independently detect and resolve conflicts using the header flags without requiring external locking or coordination. The Fence flag mechanism allows processes to autonomously determine whether to proceed with or abort an insertion based on the current state of entry headers
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to one or more embodiments, lookup, insertion, and deletion operations are allowed to continue during actions required for collision remediation. When relocation operations are used to resolve a collision, information encoded in header portions of the hash table entries that store the key-value pairs indicates when the associated key-value pairs are undergoing relocation. This information facilitates continued access to the RKVS during the relocation process by allowing other processes that access the RKVS to handle relocations without failure. Furthermore, when hash table expansion is needed in order to resolve a collision, a second, larger, hash table is allocated, and lookup operations continue on both the old hash table and the new hash table. One or more embodiments further prevent insertion, lookup, and deletion failures in the RKVS using flags, encoded in header information in hash table entries, that reflect the state of the respective key-value pairs in the store.