RDMA Key-Value Indexing With Continuous Memory Range Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Key-value databases experience high communication overheads and low query efficiency due to one-to-one query operations, which are not supported by RDMA technology.
Innovation Solution
Implement a data processing method that utilizes first and second index information to store key-value pairs and groups in continuous memory segments, enabling range queries and reducing communication overheads by allowing multiple key-value pairs to be queried based on one keyword.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If one-to-one query operations are used in key-value database, then simple query structure is maintained, but communication overheads increase and query efficiency decreases
Solution Approach 1:
The patent segments the key-value storage into multiple index levels: first index information stores individual key-value pairs, while second index information groups multiple keywords into keyword groups with corresponding value groups. This segmentation enables range queries to efficiently retrieve multiple key-value pairs by accessing grouped data structures, thereby improving query efficiency without significantly complicating the overall system architecture.
Solution Approach 2:
The patent introduces a new dimension of organization by creating grouped structures (keyword groups and value groups) that aggregate multiple individual key-value pairs. This dimensional transformation from individual units to grouped units enables batch processing and range queries, allowing the system to retrieve multiple values in a single operation rather than performing multiple separate queries.
2Ease of operation
If one-to-one query operations are used in key-value database, then individual key-value pair access is simple, but communication overheads increase
Solution Approach 1:
The patent merges multiple individual key-value pairs into grouped structures where multiple keywords are associated with multiple values in a single data structure. This merging allows the system to retrieve multiple key-value pairs through a single communication operation between client and server, significantly reducing communication overheads compared to performing separate one-to-one queries for each key-value pair.
3Adaptability or versatility
If range query is implemented using traditional key-value storage, then multiple values can be queried, but data is not stored in continuous space leading to inefficiency
Solution Approach 1:
The patent segments data into continuous storage spaces where value groups are stored adjacently in memory. This segmentation creates dedicated continuous regions for grouped data, enabling efficient sequential access during range queries. The continuous storage arrangement allows the system to retrieve multiple values without scattered memory access, thereby improving data access efficiency while maintaining range query capability.
Data Source
AI summary
One example server includes a network interface card and a memory. The memory is configured to store first index information and second index information. The network interface card is configured to receive an operation request, and access data in the memory based on the first index information and the second index information. The first index information includes a plurality of key-value pairs, and one key-value pair includes a keyword and a value. The second index information includes a plurality of key-value pair groups, and one key-value pair group includes a keyword group and a value group. The keyword group corresponds to a plurality of keywords in the first index information, and the value group corresponds to a plurality of values in the first index information. The keyword group and the value group included in the key-value pair group are stored in a segment of continuous storage space of the memory.