Database Index Contention Under High Concurrency

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Database systems experience significant slowdowns during high concurrency due to index contention, particularly when updating monotonically increasing indexed column values, as conventional approaches like sharding are costly, complex, and do not fully address the issue.

Innovation Solution

The technique involves spreading new index values over a localized range to break monotonicity, reducing contention by distributing writes across multiple nodes in the B-tree data structure, thus eliminating the need for sharding and maintaining performance without requiring changes to user queries or additional hardware.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If database indexes are updated with monotonically increasing values during high concurrency, then data retrieval accuracy is maintained, but write throughput deteriorates due to index contention

Engineering Contradiction:
Improvedata retrieval accuracyVSAvoidwrite throughput
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent segments the index update operations by dividing the index structure into multiple segments or ranges. Instead of updating the entire index sequentially with monotonically increasing values, the system divides index updates into parallel segments that can be processed concurrently, thereby maintaining data retrieval accuracy while improving write throughput during high concurrency periods.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a new dimension to index value management by using non-monotonic index values that incorporate multiple dimensions (such as combining sequential counters with hash-based distributions or using multi-level indexing). This allows the system to maintain accurate data retrieval by preserving logical ordering while enabling parallel updates through dimensional decomposition of the index space.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Productivity

If conventional approaches like sharding are used to address index contention, then write throughput is improved, but system complexity increases

Engineering Contradiction:
Improvewrite throughputVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent implements a universal index update mechanism that handles both high concurrency write operations and accurate data retrieval through a single integrated approach. The system uses a multi-functional index structure that simultaneously supports parallel updates and efficient queries without requiring separate sharding infrastructure, thereby improving write throughput while avoiding the complexity of multiple specialized systems.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent uses copying techniques by maintaining multiple copies or representations of index data in different formats or locations. Instead of physically sharding the database, the system creates virtual copies of index structures that can be updated in parallel while presenting a unified interface to users, thus improving throughput without increasing actual system complexity.

Inventive Principle:
Principle #26Copying

3Productivity

If index values are spread non-monotonically to reduce contention, then write throughput is improved, but query performance may deteriorate

Engineering Contradiction:
Improvewrite throughputVSAvoidquery performance
Core Design Contradiction:
ProductivityVSSpeed

Solution Approach 1:

The patent replaces the mechanical sequential ordering mechanism with a computational approach. Instead of relying on physical monotonic ordering of index values for query performance, the system uses computational algorithms (such as hash functions, bitmap indexes, or inverted indexes) that can handle non-monotonic values while maintaining efficient query performance through software-based optimization rather than mechanical ordering.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the parameters of the index structure by allowing index values to be non-monotonic while compensating through additional parameters or metadata. The system modifies the index to include auxiliary information (such as position markers, range boundaries, or transformation functions) that enables efficient queries even when the primary index values are distributed non-monotonically, thus improving write throughput without sacrificing query performance.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS11366810B2Index contention under high concurrency in a database system
Publication Date: 2022.06.21 SALESFORCE INC
  • US11366810B2 patent drawing
  • US11366810B2 patent drawing
  • US11366810B2 patent drawing

AI summary

A database query message may be received at a database system via a network. The database query message may include one or more parameters for retrieving data from a database based on a designated database record insertion timestamp. A database query may be executed against the database system to retrieve one or more database records based on a database record insertion timestamp range determined based on the designated database record insertion timestamp and including a plurality of database record insertion timestamp values.