Lock-Free Transaction Coordinator for In-Memory Database Consistency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In distributed database transactions, consistency issues arise when one or more databases go offline, leading to pending or aborted states, database corruption, and data mismatches due to the lack of atomic operation across all involved databases.

Innovation Solution

A transaction coordinator system that tracks the status of distributed transactions by receiving reports from participants, determining the outcome based on participant reports, and ensuring that transactions are either fully committed or aborted across all participants, using a lock-free data structure to manage and coordinate distributed transactions efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If each database maintains its own independent transaction log, then system scalability and independence are improved, but transaction consistency and atomicity deteriorate when databases go offline or fail

Engineering Contradiction:

Improvesystem scalabilityVSAvoidtransaction consistency

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

A transaction coordinator is introduced as an intermediary component that mediates between multiple databases. The coordinator maintains a global view of transaction status across all databases, receives participant reports, determines transaction outcomes, and ensures atomicity by coordinating commit/abort decisions centrally, thus resolving the consistency issue while preserving scalability

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system is segmented into independent database participants that maintain their own transaction logs for scalability, while a separate coordination layer handles atomicity. This segmentation allows databases to operate independently yet remain coordinated through the transaction coordinator, resolving the contradiction between independence and consistency

Inventive Principle:

Principle #1Segmentation

2Reliability

If distributed transactions require atomic execution across all databases, then data consistency is improved, but system complexity and coordination overhead increase

Engineering Contradiction:

Improvedata consistencyVSAvoidcoordination complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Database participants autonomously generate and transmit their own participant reports to the transaction coordinator without external intervention. Each database independently determines its local transaction status and communicates this to the coordinator, reducing coordination complexity while maintaining atomicity through self-service reporting

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system implements a feedback mechanism where participants continuously report their transaction status to the coordinator, which then determines the overall transaction outcome and communicates decisions back to participants. This feedback loop enables simple, automated coordination of atomic execution without complex manual intervention

Inventive Principle:

Principle #23Feedback

3Measurement precision

If the transaction coordinator tracks all participant status in a centralized table, then transaction outcome determination is improved, but system performance and concurrency deteriorate due to locking requirements

Engineering Contradiction:

Improvetransaction status tracking accuracyVSAvoidtransaction processing speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent replaces traditional locking mechanisms with lock-free data structures for tracking participant status. This substitution eliminates the performance bottleneck associated with locks while maintaining accurate tracking of transaction states, allowing high-concurrency operations without sacrificing measurement precision

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system uses dynamic, lock-free data structures that can be updated by multiple participants simultaneously without blocking. The transaction coordinator dynamically receives and processes participant reports in real-time, improving both accuracy of status tracking and processing speed by eliminating locking overhead

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11874816B2Lock free distributed transaction coordinator for in-memory database participants

Publication Date: 2024.01.16 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Methods, systems, apparatuses, and computer program products are provided for coordinating a distributed database transaction. A transaction driver, such as a client machine, may initiate a distributed transaction. The transaction driver may transmit to a transaction coordinator a driver report that includes identifying information related to the distributed transaction, including an identification of participants involved in the transaction. The coordinator may determine whether participant reports, which include a status of the portion of the distributed database transaction of a particular participant, are received from each of the participants. Participant reports may also identify participants that are descendants of the reporting participant. The transaction coordinator may store, in a table, information to track the progress of the distributed transaction. Using the table, an outcome of the distributed transaction may be determined and transmitted to one or more of the involved participants.