Memory-Mapped File Crash Consistency via Undo Log
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data protection systems require system shutdown during backups, limit recovery points in time, and have lengthy data recovery processes, failing to provide continuous data protection with minimal downtime and precise recovery capabilities.
Innovation Solution
A computer-implemented method using a memory-mapped file system that enables paging to non-volatile storage, managing write and read IO operations through an undo log to create crash-consistent checkpoints, allowing for efficient data recovery and minimal downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional tape backup systems are used, then data protection is provided, but system shutdown is required during backup and recovery time is limited
Solution Approach 1:
The patent implements continuous data protection by maintaining a memory-mapped file that is continuously paged to non-volatile storage, allowing data to be protected without system shutdown. The consistency layer continuously tracks write operations and maintains crash-consistent checkpoints, enabling recovery at any point in time without requiring periodic shutdowns for backups.
Solution Approach 2:
The patent performs preliminary actions by pre-establishing a memory-mapped file structure and consistency layer before data operations occur. The system pre-configures paging mechanisms and undo log structures, so that when data needs to be protected, the infrastructure is already in place to handle crash-consistent checkpoints without requiring system shutdown.
2Reliability
If daily backup is performed, then data is protected, but several hours of lost data occur between backup points
Solution Approach 1:
The patent transitions from periodic daily backups to continuous data protection through a memory-mapped file system that continuously pages data to non-volatile storage. The consistency layer continuously monitors and logs write operations, enabling recovery at any precise moment in time rather than being limited to discrete backup intervals, thus eliminating hours of potential data loss between backup points.
Solution Approach 2:
The system pre-establishes a continuous logging mechanism through the consistency layer that records all write operations to the memory-mapped file. This preliminary action of continuous logging ensures that when a crash occurs, the system has already captured the state of data at the exact moment of failure, enabling precise recovery to any point in time.
3Reliability
If conventional backup systems are used, then data is backed up, but recovery process takes a long time
Solution Approach 1:
The patent performs preliminary actions by maintaining a memory-mapped file with crash-consistent checkpoints that are continuously updated. The consistency layer pre-logs all write operations, so when recovery is needed, the system already has the necessary state information readily available. This eliminates the need to reconstruct data from scratch during recovery, significantly reducing recovery time.
Solution Approach 2:
The patent creates a copy of the data state through the memory-mapped file system that mirrors the actual data storage. The consistency layer maintains a parallel copy of write operations in the undo log, enabling rapid recovery by copying the state from the memory-mapped file rather than reconstructing data from original sources, thus dramatically improving recovery speed.
4Volume of moving object
If memory-mapped file with paging is used, then server memory capacity is extended, but data consistency during crash must be maintained
Solution Approach 1:
The patent introduces a consistency layer as an intermediary between the memory-mapped file system and the non-volatile storage. This intermediary component tracks all write operations to the memory-mapped file and maintains crash-consistent checkpoints by logging changes in the undo log. The consistency layer mediates between the extended memory capacity provided by paging and the requirement for data consistency, ensuring that crashes result in consistent state recovery.
Solution Approach 2:
The system performs preliminary actions by pre-establishing the consistency layer and undo log structure before data operations occur. The consistency layer continuously monitors write operations to the memory-mapped file and pre-logs them in the undo log, so that when a crash occurs, the data consistency is already preserved through the pre-established logging mechanism, maintaining reliability despite extended memory capacity.
Data Source
AI summary
A computer implemented method, system, and program product for data consistency comprising establishing a memory-mapped file, enabling the memory-mapped file data to be paged to the non-volatile storage medium, receiving, at a consistency layer, a write IO; wherein the write IO corresponds to a space on the storage medium, issuing a read IO to the storage medium, wherein the read IO corresponds to the space on the storage medium to be overwritten by the write IO, writing the data of the read IO to a undo log and writing the write IO to the storage medium.