Cascading Index Structure Reduces Disk I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing index structures, such as B-trees, face inefficiencies in maintenance and query performance due to high disk I/O operations, especially with increasing data volumes and the need for real-time indexing, which requires frequent updates and random disk access, leading to performance bottlenecks and inefficiencies in maintaining current data.
Innovation Solution
The implementation of a cascading index structure that includes a base level index and upper-level journal indexes, where updates are accumulated in journal indexes before being merged into the main index, reducing disk I/O operations by using sequential storage and compression techniques, and employing a Bloom filter for query efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a B-tree index structure is updated in real-time for each addition or deletion to an indexed object, then the index remains current and reflects latest data changes, but the number of disk I/O operations increases significantly (at least two disk I/O operations per update)
Solution Approach 1:
The patent divides the single B-tree index into multiple B-tree indexes organized in a cascading structure with a root B-tree and multiple child B-trees. This segmentation allows updates to be distributed across multiple smaller indexes rather than concentrated in one large index, reducing the disk I/O overhead for each individual update operation while maintaining real-time index currentness.
Solution Approach 2:
The patent introduces a hierarchical dimension to the traditional flat B-tree structure by creating a cascading multi-level index system. The root B-tree references multiple child B-trees, adding a vertical dimension to the index organization. This dimensional change enables more efficient update operations by allowing selective updates to specific child B-trees based on the nature of the data change.
2Adaptability or versatility
If multiple B-trees are generated on a single indexed object to index different columns, then query flexibility improves, but update operations become more expensive due to random disk I/O across multiple leaf blocks
Solution Approach 1:
The patent merges multiple B-tree indexes into a cascading structure where a root B-tree coordinates multiple child B-trees. This merging approach maintains the query flexibility of having multiple indexes while reducing update costs by allowing coordinated updates across the merged structure. The root B-tree manages the coordination, reducing random disk I/O compared to independently updating multiple separate B-trees.
Solution Approach 2:
The root B-tree serves multiple functions: it acts as an index for root-level columns and simultaneously provides a coordination structure for managing multiple child B-trees. This multi-functionality reduces the need for separate update mechanisms for each B-tree, thereby reducing overall update operation costs while preserving query flexibility.
3Productivity
If batch updates are performed during off-peak hours to reduce real-time disk I/O, then disk I/O operations decrease, but the index is not current and query performance degrades
Solution Approach 1:
The patent implements periodic merging operations where child B-trees are merged into the root B-tree at regular intervals or based on accumulated update thresholds. This periodic action allows the system to operate in a relaxed mode during intervals (reducing real-time disk I/O) while periodically synchronizing to maintain index currentness, thus resolving the contradiction between reducing disk I/O and maintaining up-to-date indexes.
Solution Approach 2:
The patent performs preliminary updates in child B-trees before merging into the root B-tree. This preliminary action allows updates to be staged in memory or in less critical storage locations first, reducing immediate disk I/O pressure, and then merged periodically into the main index structure to ensure currentness is maintained without continuous high-cost disk operations.
Data Source
AI summary
Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to storage as a result of the merge operation.