Database Search Index Sub-Index Segmentation and Pointer Swapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large databases face inefficiencies in indexing and maintenance due to incremental builds which can take hours or days, leading to infrequent updates and inaccuracies, making them inoperable during the process.
Innovation Solution
The system detects an incremental build trigger and builds new sub-indexes to replace affected ones, using old sub-indexes during generation and moving pointers only when new sub-indexes are ready, allowing seamless database queries and maintenance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If incremental index builds are performed frequently to maintain accuracy, then database accuracy is improved, but database availability deteriorates because the database becomes inoperable during the build process
Solution Approach 1:
The index is divided into multiple sub-indices, each maintained independently. When a sub-index needs updating, only that specific sub-index is rebuilt while others remain operational, allowing the database to remain available during index maintenance operations
Solution Approach 2:
The system builds new sub-indices alongside existing ones, then switches to the new sub-indices once complete. The old sub-indices are discarded after successful replacement, enabling continuous operation without downtime while maintaining accurate indexing
2Measurement precision
If incremental index builds are performed to update the database index, then database accuracy is improved, but operational downtime increases making the database inoperable
Solution Approach 1:
The system performs index maintenance in advance by building new sub-indices before they are needed, or in parallel with existing ones. This preliminary action ensures that accurate indices are ready before switchover, eliminating delays and keeping the database operational throughout the process
Solution Approach 2:
The system uses pointer mechanisms as intermediaries to switch between old and new sub-indices. The pointers can be updated atomically to point to the newly built sub-indices, allowing seamless transition without interrupting database operations or causing downtime
3Measurement precision
If the entire index is rebuilt to ensure accuracy, then database accuracy is improved, but the time and resources required increase significantly
Solution Approach 1:
The large index is segmented into multiple smaller sub-indices. Only the specific sub-indices that have changed or need updating are rebuilt, rather than reconstructing the entire index. This segmentation dramatically reduces the time and computational resources required while maintaining overall index accuracy
Solution Approach 2:
The system performs partial index builds only on the affected sub-indices rather than complete index reconstruction. This partial action is sufficient to maintain accuracy for changed data while avoiding the excessive time and resource costs of full index rebuilds
Data Source
AI summary
Methods and apparatus for building a search index for a database are disclosed. When an incremental build trigger is detected (e.g., a threshold number of documents are added to database), the system determines which sub-indexes need to be updated and which sub-indexes do not need to be updated. Rather than update the affected sub-indexes directly, the system builds new sub-indexes to replace the affected sub-indexes. Database queries that occur during the generation of the replacement sub-indexes use the old sub-indexes. When the new sub-indexes are ready, the system moves pointers from the old sub-indexes to the new sub-indexes so that subsequent database queries use the new sub-indexes.