Reordering Enriched Inverted Indices for Database Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The rebuilding of enriched inverted indices in databases is computationally expensive due to the reassignment of document identifiers, which requires updating the indices to maintain optimal search performance and compression rates, especially with changes in database content such as additions, removals, or updates of records.

Innovation Solution

The enriched inverted index is reordered based on reassigned document identifiers, with the reordering performed locally on individual components like inverted lists and enrichment data structures, allowing for parallel or sequential reordering to maintain optimal document identifier assignment and improve search performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If full index rebuilding is performed to maintain optimal search performance after document identifier reassignment, then search performance is improved, but computational overhead and processing time increase significantly

Engineering Contradiction:

Improvesearch performanceVSAvoidcomputational overhead

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the index rebuilding process into two independent phases: first reordering the inverted lists containing document identifiers, and then reordering the enrichment data structures. This segmentation allows each phase to be optimized independently and reduces the overall computational burden compared to rebuilding the entire index from scratch.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary reordering of document identifiers in inverted lists before reordering enrichment data. This preliminary action establishes a new baseline order that subsequent enrichment data reordering can reference, avoiding the need to rebuild the entire index structure and reducing computational overhead while maintaining search performance.

Inventive Principle:

Principle #10Preliminary action

2Loss of time

If enriched inverted index is reordered locally on individual components to reduce computational overhead, then processing time is reduced, but maintaining consistency between inverted lists and enrichment data becomes more complex

Engineering Contradiction:

Improveprocessing timeVSAvoidindex reordering complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent divides the enriched inverted index into separable components: inverted lists storing document identifiers and enrichment data structures storing additional information. Each component is reordered independently using the same reordering logic, which simplifies the implementation complexity while maintaining consistency across the entire index structure.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the ordering parameter of document identifiers in inverted lists and applies the same parameter change to the corresponding enrichment data structures. This parameter-based reordering approach maintains consistency between components while reducing processing time, as the same reordering logic can be applied uniformly across different data structures.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If document identifiers are reassigned to optimize compression rates and search performance, then database efficiency is improved, but the computational cost of updating all index components increases

Engineering Contradiction:

Improvedatabase efficiencyVSAvoidcomputational cost

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent performs partial reordering of index components rather than complete rebuilding. By reordering only the inverted lists and then reordering enrichment data based on the new inverted list order, the patent achieves the necessary document identifier reassignment for optimized compression and search performance while performing less computational work than a full index rebuild would require.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS10452693B2Reordering of enriched inverted indices

Publication Date: 2019.10.22 SAP SE

AI summary

A method can include: reordering an enriched inverted index associated with a database, the enriched inverted index including a first inverted list having a first plurality of current document identifiers of records that contain a first data value, the enriched inverted index further including a first data structure storing enrichment data, the reordering of the enriched inverted index comprising: generating an ordinal sequence corresponding to an order of a first plurality of current document identifiers that include a change of at least one of the first plurality of current document identifiers to a new document identifier; determining a reordered ordinal sequence corresponding to a sorted order of the second plurality of document identifiers; separately reordering, based at least on the reordered ordinal sequence, the first plurality of current document identifiers in the first inverted list and the enrichment data in the first data structure.