Secondary Index Creation Using Range Locking for Data Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Creating secondary indexes in database systems can be resource-intensive and lead to data inconsistency when done online, as it requires accessing and rearranging data while still allowing access to the table, which can result in stale or inconsistent data being stored in the index.
Innovation Solution
Implementing locking item ranges allows for the creation of secondary indexes from online tables by locking specific ranges of items, delaying updates within the locked range until the index is generated, ensuring data consistency without incurring significant additional operational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If secondary indexes are created from online tables without locking, then system availability is maintained, but data consistency deteriorates
Solution Approach 1:
The patent segments the table into multiple item ranges and locks only the specific ranges being indexed rather than the entire table. This allows other parts of the table to remain accessible, maintaining system availability while ensuring data consistency for the locked ranges during index creation.
Solution Approach 2:
The patent applies locking selectively to local portions (specific item ranges) of the table rather than globally. This local quality approach ensures data consistency where needed (locked ranges) while preserving system availability in other areas (unlocked ranges).
2Reliability
If the entire table is locked for index creation, then data consistency is ensured, but system performance deteriorates
Solution Approach 1:
The patent divides the table into multiple item ranges and applies locking only to the specific ranges being indexed. This segmentation reduces the scope of locking from the entire table to just the necessary portions, maintaining data consistency while preserving system performance through continued access to unlocked ranges.
Solution Approach 2:
The patent applies partial locking action by locking only the specific item ranges needed for index creation rather than the entire table. This partial action is sufficient to ensure data consistency for the indexed portions while minimizing the impact on overall system performance.
Data Source
AI summary
A data storage system may implement locking item ranges for creating a secondary index of an online table. A secondary index may be generated for a table of items stored in a non-relational data store. Different ranges of items in the data store may be locked while a corresponding portion of the secondary index is generated. Upon generating the corresponding portion of the secondary index, a range of items may be unlocked. While generating the secondary index, the table may be made available for servicing access requests. For a request to update the table received during the generation of the secondary index, a determination may be made as to whether the update is included within a locked range of the table. If locked, the request may be delayed until the range is unlocked.