RDMA Linear Hash Probing via Dynamic Slot Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In densely populated linear hash tables accessed via RDMA, the need to read through numerous slots to find an empty or target slot leads to inefficiencies due to either excessive data retrieval or high latency from multiple RDMA requests, as the optimal read size is often unknown.
Innovation Solution
A system dynamically predicts the number of slots to read in an RDMA request by determining probabilities of reaching an empty slot and using these probabilities to minimize the cost of probing operations, adjusting read sizes based on hash table occupancy and network thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system reads a large number of slots in each RDMA operation, then the probability of finding the target or empty slot increases, but network overhead and data retrieval volume increase
Solution Approach 1:
The patent implements dynamic read size adjustment where the RDMA read size is not fixed but adapts based on the current state of the linear hash table. The system calculates the expected number of probes needed and adjusts the read size accordingly, making the system flexible and responsive to changing conditions rather than using a static configuration.
Solution Approach 2:
The patent changes the parameter of RDMA read size based on calculated probabilities and expected probe counts. By computing the optimal read size as a function of hash table occupancy and probe expectations, the system dynamically adjusts this critical parameter to balance between finding the target slot efficiently and minimizing unnecessary data transfer.
2Reliability
If the system uses multiple RDMA requests to read through slots, then it can find the target or empty slot, but latency increases
Solution Approach 1:
The patent performs preliminary calculation of the expected probe count and optimal read size before executing the RDMA operations. By pre-computing these values based on the current hash table state and probability models, the system prepares the optimal strategy in advance, avoiding the need for multiple sequential RDMA requests and reducing overall latency.
3Loss of energy
If the system reads fewer slots per RDMA operation, then network overhead decreases, but the number of RDMA requests increases
Solution Approach 1:
The patent incorporates feedback mechanisms where the system monitors the actual performance and state of the linear hash table during probing operations. This feedback is used to refine the probability models and adjust subsequent read sizes, creating a closed-loop system that continuously optimizes the balance between network overhead and probing efficiency based on observed conditions.
Data Source
AI summary
Systems and methods for reducing latency of probing operations of remotely located linear hash tables are described herein. In an embodiment, a system receives a request to perform a probing operation on a remotely located linear hash table based on a key value. Prior to performing the probing operation, the system dynamically predicts a number of slots for a single read of the linear hash table to minimize total cost for an average probing operation. The system determines a hash value based on the key value and determines a slot of the linear hash table to which the hash value corresponds. After predicting the number of slots, the system issues an RDMA request to perform a read of the predicted number of slots from the linear hash table starting at the slot to which the hash value corresponds.