Multi-Layer Data Index Segmentation for Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data index construction methods require a fixed rule and occupy significant storage space, limiting their efficiency and adaptability to data characteristics.
Innovation Solution
A data index establishment method that segments raw data based on key characteristics, using monotone functions to create mapping relationships that are stored efficiently, allowing for multi-layer indexing to improve search efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a B+ tree index is constructed according to a fixed rule, then data search efficiency is improved, but the index occupies a large amount of storage space
Solution Approach 1:
The patent segments the raw data table into multiple data segments based on key ranges, and constructs separate index structures for each segment. This segmentation allows the index to only store mappings for its respective segment rather than the entire dataset, significantly reducing the storage space required for each index while maintaining efficient search capability within each segment.
2Speed
If a B+ tree index is constructed according to a fixed rule, then data search efficiency is improved, but the index construction complexity increases
Solution Approach 1:
The patent applies different index construction strategies to different data segments based on their local characteristics. Each data segment can have its own optimized index structure tailored to its specific key range and data distribution, rather than forcing a uniform index structure across all data. This local optimization reduces overall construction complexity while maintaining search efficiency.
3Productivity
If an index is constructed to reflect data characteristics, then query efficiency is improved, but the index structure becomes more complex
Solution Approach 1:
The patent introduces a hierarchical dimension to the index structure by creating multiple levels of indexing. The first level divides the data into segments, and subsequent levels provide further refinement within each segment. This dimensional approach allows the index to adapt to data characteristics at different granularities, improving query efficiency for various types of queries while keeping each individual index level relatively simple.
Data Source
AI summary
A data index establishment method obtains M raw data segments and a first-layer mapping relationship through segment fitting on raw data in a raw data table, and further obtains N first key segments and a second-layer mapping relationship through segment fitting on a first key of each raw data segment. In a subsequent data query process, a multi-layer mapping relationship is input from a to-be-queried key sequentially in descending order, until a location of the to-be-queried key in the raw data table is located.