HTAP Database Backup and Restore with Mutation Log Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based data warehouses face challenges in managing transactional and analytical processing efficiently, leading to data integrity issues and increased overhead costs due to separate OLTP and OLAP databases, which hinder real-time handling of time-sensitive workloads.
Innovation Solution
A hybrid transactional/analytical processing (HTAP) system integrates OLTP and OLAP workloads within a single database, minimizing the need for ETL pipelines and optimizing concurrent transactions using a concurrency control mechanism with MVCC and locking, along with a two-level transaction hierarchy and distributed deadlock detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate OLTP and OLAP databases are used, then transactional processing and analytical processing can be handled independently, but overhead costs increase and data integrity issues arise
Solution Approach 1:
The patent combines OLTP and OLAP databases into a single unified database system, eliminating the need for separate database infrastructures. This merger reduces overhead costs and simplifies data management while maintaining data integrity through a shared data model and unified transaction management mechanisms.
2Productivity
If ETL pipelines are used to transfer data between OLTP and OLAP, then data can be processed, but the process is time-consuming and affects real-time handling
Solution Approach 1:
The patent extracts and eliminates the ETL pipeline component from the system architecture. By integrating analytical processing directly into the transactional database, data no longer needs to be transferred between separate systems, enabling real-time processing without the time-consuming ETL extraction, transformation, and loading steps.
3Productivity
If a unified HTAP database is implemented, then real-time handling of time-sensitive workloads is improved, but concurrency control complexity increases
Solution Approach 1:
The patent segments concurrency control into distinct isolation levels (read committed, snapshot isolation, serializable) with specific mechanisms for each. This segmentation allows the system to handle concurrent transactions efficiently by applying appropriate control strategies based on the required isolation level, managing complexity through structured categorization rather than a monolithic approach.
4Adaptability or versatility
If transactional processing is expanded in cloud-based data warehouses, then more operations are available, but data compaction becomes difficult and security is compromised
Solution Approach 1:
The patent implements dynamic data compaction mechanisms that automatically adjust to transactional workload characteristics. The system employs real-time data organization and compression techniques that adapt to changing data patterns, maintaining security and integrity while enabling efficient storage of expanded transactional data volumes without manual intervention.
Data Source
AI summary
The subject technology initiates a reinstallation process of a key-value storage device and locking the key-value storage device. The subject technology performs a bootstrap process for a blob manager and a blob worker. The subject technology performs a restoration process of a storage server. The subject technology applies a set of mutation logs to the storage server. The subject technology unlocks the key-value storage device and enabling network traffic for the key-value storage device.