Multi-version B-tree Writeable Snapshots for OLTP Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems face challenges in efficiently executing both online transaction processing (OLTP) and analytics transactions concurrently, leading to high latency and performance issues due to the mix of transactions that apply insertions and updates, with data often being stale or out-of-date, which affects analysis and evaluation quality.
Innovation Solution
A data management system utilizing a multi-version B-tree with writeable snapshots and optimistic concurrency control, allowing for point-in-time views of data, enabling simultaneous execution of OLTP and analytics transactions without disrupting performance, and supporting efficient archiving, continuous data protection, and fine-grained control over roll-backs and recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a traditional single-version database system is used to execute both OLTP and analytics transactions, then the system structure remains simple, but the analytics transactions experience high latency and performance degradation due to conflicts with OLTP transactions and access to stale data
Solution Approach 1:
The patent segments the database system into multiple independent versions, each representing a specific point in time. Analytics transactions can access historical versions while OLTP transactions operate on the current version, eliminating performance conflicts. Each version is an independent copy of the data structure, allowing parallel access without interference.
Solution Approach 2:
The patent creates multiple copies of the data structure (versions) to enable concurrent analytics queries on historical data without affecting OLTP operations. Each version is a complete copy that can be independently queried, allowing analytics to run on stale data while OLTP works on current data, thus resolving the performance contradiction.
2Productivity
If multiple versions of data are maintained to support analytics transactions, then analytics performance improves, but the storage requirements and system complexity increase
Solution Approach 1:
The patent implements a nested version structure where older versions are retained but can be pruned when superseded by newer versions. Each version contains the complete data structure, but the system manages them in a nested hierarchy where Version N contains all data that Version N-1 had plus additional updates. This allows efficient storage management where pruned versions free up space while maintaining necessary historical data.
Solution Approach 2:
The patent implements automatic pruning of old versions that are no longer needed for analytics. When a new version is created, older versions that have been superseded can be discarded to recover storage space. This dynamic management allows the system to maintain multiple versions for concurrent access while automatically recovering storage resources by discarding obsolete versions.
3Reliability
If analytics transactions access current data to improve relevance, then data freshness improves, but OLTP transaction performance degrades due to resource contention
Solution Approach 1:
The patent implements dynamic version selection where analytics transactions can choose to access either current or historical versions based on their specific needs. The system dynamically assigns analytics queries to appropriate versions, allowing some analytics to use current data when available while others use historical versions, thus balancing data freshness requirements with OLTP performance requirements.
Solution Approach 2:
The patent introduces version metadata and a version management layer that acts as an intermediary between analytics queries and the actual data. This intermediary layer routes analytics transactions to appropriate versions based on their requirements, preventing direct contention with OLTP transactions on current data while still providing access to fresh data when appropriate.
4Measurement precision
If frequent snapshots are taken to improve data accuracy for analytics, then analysis quality improves, but the overhead for creating and managing snapshots increases
Solution Approach 1:
The patent creates data versions in advance of when they are needed for analytics. Each OLTP transaction that modifies data automatically creates a new version, so when analytics queries need historical data, the versions are already prepared and available. This eliminates the need for separate snapshot creation operations at query time, reducing overhead while maintaining data accuracy.
Solution Approach 2:
The patent implements automatic version creation as part of the normal OLTP transaction process. Each commit operation automatically generates a new version without requiring separate snapshot operations. This self-service approach means versions are created incidentally during regular data updates, eliminating dedicated snapshot creation time while ensuring data accuracy for analytics.
Data Source
AI summary
A method for managing data using writeable snapshots in a multi-versioned, distributed B-tree comprising nodes distributed over a plurality of servers, includes receiving a transaction request specifying a transaction type, the transaction having a transaction snapshot id, and determining the transaction type. If the transaction type is an update transaction, the method includes determining a node to update, which is defined by a node snapshot id, and if the transaction snapshot id is greater that the node snapshot id, the method copies the node and updates the copy to create a writeable snapshot, assigns the transaction snapshot id as a node snapshot id of the copy, and places a pointer from the node to the copy.