Shard Index Selection for Lower-Cost Scalable Database Retrieval

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Replicating large amounts of data in databases results in high costs for building secondary indexes, which are necessary for data retrieval, especially in multi-primary and primary-secondary configurations.

Innovation Solution

Implement scalable indexes by selecting one or more shard copies based on access usage, such as access frequency and patterns, and building indexes only for these selected copies, rather than for all shard copies.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If indexes are built for all shard copies to improve data retrieval efficiency, then data retrieval speed is improved, but storage and processing costs increase significantly

Engineering Contradiction:

Improvedata retrieval speedVSAvoidstorage and processing costs

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The patent applies local quality by building indexes selectively on specific shard copies based on their access patterns rather than uniformly on all shards. The system identifies which shard copies are frequently accessed and builds indexes only on those, making the indexing strategy adaptive to local access characteristics and reducing overall storage and processing costs while maintaining retrieval efficiency for commonly accessed data.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent implements partial action by building indexes on only a subset of shard copies rather than all of them. The system determines the optimal number and selection of shard copies for indexing based on access patterns, performing indexing on just enough shards to maintain acceptable retrieval performance while avoiding the excessive cost of indexing every single shard copy.

Inventive Principle:

Principle #16Partial or excessive action

2Ease of operation

If indexes are built for every shard copy to ensure efficient retrieval, then data accessibility is improved, but resource utilization deteriorates due to redundant index maintenance

Engineering Contradiction:

Improvedata accessibilityVSAvoidresource utilization

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent makes resource utilization adaptive by varying the indexing strategy across different shard copies based on their access patterns. Frequently accessed shards receive indexes to ensure fast retrieval, while less accessed shards do not require indexes, thereby optimizing resource utilization dynamically rather than maintaining uniform indexing across all shards.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent introduces dynamics into the indexing strategy by continuously monitoring access patterns and adjusting which shard copies have indexes built on them. The system can add or remove indexes from shard copies based on changing access characteristics, making the resource allocation flexible and adaptive rather than static.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12517882B2Building and using scalable database indexes

Publication Date: 2026.01.06 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Access usage of one or more shards of a set of shards of a database is determined. The set of shards includes a plurality of shards including a primary shard and one or more replica shards of the primary shard. A shard of the plurality of shards is selected based on the access usage, and a set of indexes is built for the shard that is selected. The set of indexes built for the shard that is selected is used for at least multiple shards of the plurality of shards.