Commit Index Mapping for Distributed Storage Read Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, existing data reading methods are inefficient due to the need to wait for write operations to complete before reading data, especially for applications with more reads than writes, leading to delayed data access and reduced user experience.
Innovation Solution
A data reading method that utilizes a dedicated mapping table to set and update commit indices, allowing data to be read from replicas with an apply index not smaller than the target commit index, even if the target application has not performed a write operation, thereby improving data retrieval efficiency and reducing delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data reading waits for write operations to complete before reading, then data consistency is ensured, but data reading efficiency deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-establishing commit indices in the mapping table during write operations, so that when read operations occur, the necessary index information is already prepared and available immediately without waiting for write completion
Solution Approach 2:
The mapping table acts as an intermediary component that stores commit indices between write operations and read operations, enabling read operations to proceed independently of write operation completion status while maintaining data consistency through the indexed information
2Reliability
If applications with more reads than writes wait for write operations, then data consistency is maintained, but user experience deteriorates
Solution Approach 1:
The mapping table stores commit indices in advance during write operations, enabling read operations to immediately query and retrieve data without waiting for write completion, thus eliminating access delay for read-heavy applications
Solution Approach 2:
The system uses feedback from write operations (commit indices) to enable immediate read operations, creating a feedback loop where write information is stored and reused by read operations without waiting for completion
Data Source
AI summary
The present disclosure provides a data reading method, including: in response to receiving a read response request generated by a replication group for an application and sent by a storage terminal, setting, in a dedicated mapping table corresponding to the application, a commit index corresponding to the replication group as a commit index carried by the read response request; searching for a target replication group corresponding to a first read request generated by a target application based on the first read request; determining, in a dedicated mapping table corresponding to the target application, a commit index corresponding to the target replication group as a target commit index; sending a second read request carrying the target commit index to the storage terminal; and obtaining the data read by the storage terminal.