Blind Update Mechanism for Key-Value Store Index Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing key-value store systems face bottlenecks in index insertion throughput due to the need for read operations before writes, particularly in applications like inverted indexes, where updates do not require knowledge of the existing value, and log-structured merge trees that lack semantic value merge support, leading to slowed index insertions.
Innovation Solution
The implementation of a 'blind' incremental update mechanism that allows updates to be performed without reading the logical page, using a page stub to store update information and delta records, which are then flushed to storage using incremental page flushing, enabling full storage write bandwidth utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If read operations are performed before write operations in key-value store systems, then data consistency is ensured, but index insertion throughput is bottlenecked
Solution Approach 1:
The patent applies preliminary action by performing write operations (blind incremental updates) without waiting for read operations to complete. Update requests are posted and applied to logical pages immediately, bypassing the traditional read-then-write sequence. This allows index insertions to proceed in parallel with read operations, eliminating the read operation bottleneck and achieving throughput approaching sequential storage write bandwidth.
2Productivity
If log-structured merge trees are used for storage, then storage efficiency is improved, but semantic value merge support is lacking, leading to slowed index insertions
Solution Approach 1:
The patent changes the operational parameters of log-structured merge trees by introducing blind incremental updates that operate independently of semantic value merge capabilities. Instead of requiring the storage system to understand and merge semantic values, the system posts update requests with delta records that are applied sequentially. This parameter change allows index insertions to proceed at high speed without requiring semantic value merge support, while maintaining compatibility with existing log-structured merge tree implementations.
3Productivity
If traditional update mechanisms are used that access logical pages via read operations, then data accuracy is maintained, but full storage write bandwidth cannot be utilized
Solution Approach 1:
The patent introduces an intermediary mechanism (update acquisition module and update posting engine) that decouples the update posting process from read operations. The update acquisition module obtains update requests and the update posting engine posts them to logical pages without requiring read access. This intermediary layer maintains data accuracy through proper update tracking and application while enabling full storage write bandwidth utilization by eliminating read operation dependencies.
Data Source
Figure 1A~1B
Figure 1C
Figure 2
AI summary
Update requests that specify updates to a logical page associated with a key-value store are obtained. Updates to the logical page are posted using the obtained plurality of update requests, without accessing the logical page via a read operation.