In-memory database superblock recovery with parallel encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The recovery process of in-memory databases is resource-intensive and slow due to the time-consuming process of flushing pages into physical persistence, particularly when encryption is performed on a page-by-page basis.
Innovation Solution
Combining smaller pages into superblocks and encrypting them in parallel using multiple helper threads, with encryption information included in a superblock control block, allows for efficient flushing to physical persistence, reducing recovery time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pages are encrypted on a page-by-page basis during database recovery, then security is maintained, but the recovery process becomes slow and resource-intensive
Solution Approach 1:
Multiple individual pages are merged into a single superblock structure, allowing them to be processed together as one unit. This combining approach enables parallel encryption operations across multiple pages simultaneously, improving recovery speed while maintaining security through the unified superblock encryption mechanism
Solution Approach 2:
The recovery process is segmented into parallel operations using multiple helper threads, each capable of encrypting different pages within a superblock simultaneously. This segmentation allows the system to overcome the bottleneck of single-threaded page-by-page encryption while maintaining proper security boundaries through the superblock structure
2Loss of time
If multiple helper threads are used to encrypt pages in parallel within superblocks, then database recovery time is reduced, but system complexity increases
Solution Approach 1:
The encryption workload is segmented into multiple independent helper threads that operate in parallel on different pages within a superblock. Each thread performs identical encryption operations on its assigned pages, simplifying the design while achieving parallelism and reduced recovery time
Solution Approach 2:
The superblock control block serves multiple functions: it stores encryption information for all pages within the superblock, coordinates the parallel helper threads, and manages the unified encryption process. This multi-functional design reduces overall system complexity by consolidating control mechanisms
Data Source
AI summary
Recovery of an in-memory database is initiated. Thereafter, pages for recovery having a size equal to or below a pre-defined threshold are copied to a superblock. For each copied page, encryption information is added to a superblock control block for the superblock. The copied pages are encrypted within the superblock using the corresponding encryption information added to the super block control block. The superblock is then flushed from memory (e.g., main memory, etc.) of the database to physical persistence.