Relational Database Metadata Versioning for DDL Snapshot Isolation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional relational database systems require full synchronization between read queries and data definition language operations (DDLs) due to the lack of metadata versioning, leading to performance bottlenecks, unavailability, and unexpected failures.

Innovation Solution

Implementing a relational database system with versioned metadata, a lock manager, and a transaction manager to manage metadata versions, allowing concurrent access and snapshot isolation of metadata transactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If full exclusive lock (SCH-M) is required for DDL operations to synchronize with concurrent read queries, then schema consistency is maintained, but DDL operations are blocked until all queries complete, significantly impacting execution performance and causing unavailability

Engineering Contradiction:
Improveschema consistencyVSAvoidDDL operation performance
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent segments the monolithic schema modification lock into multiple granular locks organized in a lock graph structure. Instead of requiring a single SCH-M lock on all schema objects, the system creates individual locks for each schema object and organizes them in a graph where nodes represent locks and edges represent conflict relationships. This allows concurrent DDL operations on different schema objects to proceed simultaneously without blocking each other, while still maintaining consistency through the lock graph's conflict detection mechanism.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a lock graph as an intermediary data structure that mediates between multiple DDL operations and the underlying schema objects. The lock graph tracks lock requests, determines conflicts, and coordinates grant decisions without requiring a single exclusive lock. This intermediary enables the system to maintain schema consistency while allowing multiple DDL operations to progress concurrently by resolving conflicts at the lock graph level rather than blocking at the schema object level.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If DDL operations wait for ongoing queries to complete before acquiring schema modification lock, then schema consistency is ensured, but the window of unavailability is extended and new queries are blocked

Engineering Contradiction:
Improveschema consistencyVSAvoidunavailability window
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

By segmenting the lock requirement into individual schema object locks rather than a global SCH-M lock, the system allows DDL operations to acquire locks on specific objects independently. This segmentation enables overlapping execution of DDL operations and queries on different objects, dramatically reducing the total time the system remains unavailable while maintaining consistency through the lock graph's conflict detection.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system performs preliminary lock acquisition attempts on individual schema objects before committing the DDL operation. The lock graph tracks these preliminary lock requests and determines which can be granted immediately without blocking. This preliminary action allows multiple DDL operations to make progress concurrently rather than waiting for all queries to complete before any DDL can proceed.

Inventive Principle:
Principle #10Preliminary action

3Reliability

If DDL operations require full exclusive lock on all schema objects, then schema consistency is maintained, but secondary replicas experience log replication delays and accumulate large logs on primary replica

Engineering Contradiction:
Improveschema consistencyVSAvoidlog replication speed
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent segments the schema modification process into independent lock acquisitions on individual schema objects. This segmentation allows DDL operations to proceed concurrently on different objects without requiring the primary replica to hold a global exclusive lock for the entire duration. As a result, log replication can continue for non-conflicting operations, reducing log accumulation on the primary and improving replication speed to secondary replicas.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system performs preliminary lock acquisition and conflict detection before the DDL operation commits and generates replication logs. By using the lock graph to determine which DDL operations can proceed without conflicts in advance, the system minimizes the duration of exclusive locks held on schema objects. This preliminary action reduces the window during which log replication must wait, thereby improving replication throughput and reducing log accumulation.

Inventive Principle:
Principle #10Preliminary action

4Reliability

If metadata is not versioned and full synchronization is required, then schema consistency is ensured, but query compilation fails when schema changes occur and queries must be retried

Engineering Contradiction:
Improveschema consistencyVSAvoidquery execution performance
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent segments schema consistency into object-level units tracked by the lock graph. Instead of invalidating all snapshots when any schema change occurs, the system maintains version information at the individual schema object level. This segmentation allows queries to continue using valid snapshots for the schema objects they access, even when other objects are being modified, thereby reducing query failures and retries while maintaining consistency for affected objects.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system creates and maintains copies of schema object versions in the lock graph structure. When a DDL operation modifies a schema object, the lock graph preserves the previous version information and creates new version entries. This copying mechanism allows ongoing queries to continue referencing valid historical versions of schema objects, preventing query compilation failures while still allowing the schema to evolve. Queries only need to be retried when their specific schema objects are modified, not when any schema change occurs.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS20250258811A1Schema snapshot isolation access in a relational database
Publication Date: 2025.08.14 MICROSOFT TECHNOLOGY LICENSING LLC
  • US20250258811A1 patent drawing
  • US20250258811A1 patent drawing
  • US20250258811A1 patent drawing

AI summary

Relational database systems are disclosed that are enabled to operate with versioned metadata. The relational database system includes a lock manager, a transaction manager and a version aware metadata storage and cache configured to store to store and manage versions of metadata, to determine which of such versions should be visible at any given point in time, and to enable creation of the proper versions of metadata. In an aspect, the transaction manager manages transaction identifiers and their associated start times, abort times and/or commit times. Such data enables determination of transaction visibility, and consequently the metadata version visibility, for any point in time. In an aspect, such metadata versioning support enables snapshot isolation of metadata transactions.