Host Driver Event Journaling for Zero-Downtime Data Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data protection methods, such as tape and in-storage snapshot approaches, are inadequate for real-time data protection and recovery in enterprise environments due to issues like incomplete updates, data loss, high bandwidth requirements, and manual downtime processes, which are inefficient and costly.
Innovation Solution
A data management system that provides continuous, real-time data protection by capturing and processing real-time data transactions through a host driver, creating a continuous event journal stream, allowing for automated, zero-downtime data protection and recovery across multiple data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional snapshot approaches are used to capture data, then data can be stored for later recovery, but critical data changes during snapshot capture result in incomplete updates and data inconsistency
Solution Approach 1:
The patent divides data protection into two separate components: (1) continuous event journaling that captures all data changes in real-time with timestamps, and (2) periodic snapshots that provide baseline data. This segmentation allows the system to recover complete transactions by combining journal entries with snapshot data, eliminating the inconsistency problem of traditional single-snapshot approaches.
Solution Approach 2:
The system performs preliminary action by continuously journaling all data changes before they occur, maintaining a chronological record of every transaction. This preliminary recording ensures that when recovery is needed, the complete transaction history is already captured and can be replayed to achieve consistent data restoration without worrying about changes occurring during the capture process.
2Reliability
If complete snapshot data is repeatedly offloaded and stored, then data protection is achieved, but storage device size growth makes bandwidth requirements impractical
Solution Approach 1:
The patent extracts only the essential change information from the data stream and stores it in event journals, rather than repeatedly copying complete snapshots. By taking out only the delta changes and metadata needed for recovery, the system maintains data protection while dramatically reducing the bandwidth and storage requirements compared to traditional snapshot replication.
Solution Approach 2:
The system changes the parameter of data representation from complete binary snapshots to structured event journals with metadata. This parameter change allows the system to track data changes more efficiently, storing only what is necessary for recovery (change events, timestamps, transaction identifiers) rather than redundant copies of entire data volumes, thus reducing bandwidth consumption.
3Reliability
If storage based snapshot is used, then data can be protected, but fine grain application data cannot be recovered without reintroducing the entire backup volume
Solution Approach 1:
The patent segments data into fine-grained objects (files, directories, database records) and tracks them individually through event journals. Each data object has its own change history recorded in the journal, allowing the system to recover only the specific fine-grained objects that need restoration rather than reintroducing entire backup volumes. This segmentation enables selective recovery at the object level.
Solution Approach 2:
The event journal acts as an intermediary between the data and the recovery process. Instead of directly copying data from snapshots, the system uses the journal as a mediator to identify, track, and restore individual data objects. The journal contains all necessary information (change events, timestamps, object identifiers) to reconstruct and recover specific fine-grained data objects without needing to process entire backup volumes.
4Reliability
If conventional data protection processes are used, then data backup and recovery can be performed, but significant application downtime occurs during manual operations
Solution Approach 1:
The patent implements continuity of useful action by continuously journaling all data changes in real-time while the application runs, without requiring shutdown or interruption. The event journaling process operates continuously in the background, capturing every data change as it occurs. This continuous operation ensures data protection is maintained throughout the application lifecycle, eliminating downtime associated with traditional backup and recovery operations.
Solution Approach 2:
The system enables self-service by automatically journaling all data changes without requiring manual intervention. The event processor continuously monitors data sources and records changes autonomously. During recovery, the system automatically reconstructs data objects from the journal entries without requiring manual copying or reintroduction of backup volumes. This automation eliminates the manual steps that cause downtime in conventional systems.
5Reliability
If separate and distinct processes are used for initial data upload, scheduled backup, data resynchronization, and data recovery, then each function can be performed, but many manual steps and uncoordinated systems result in significant downtime
Solution Approach 1:
The patent merges multiple separate data protection processes (initial data upload, scheduled backup, data resynchronization, and data recovery) into a single unified event journaling system. The event processor continuously performs all these functions through one coordinated mechanism: journaling data changes, maintaining baselines, enabling resynchronization, and supporting recovery. This consolidation eliminates the need for multiple uncoordinated systems and manual transition steps between different processes.
Solution Approach 2:
The event journal serves as a universal data structure that supports multiple functions simultaneously. The same journal mechanism is used for initial data capture, ongoing backup, resynchronization after failures, and recovery operations. This multi-functionality eliminates the need for separate specialized systems for each data protection task, reducing system complexity and coordination overhead while maintaining comprehensive data protection capabilities.
Data Source
AI summary
A data management system or “DMS” provides an automated, continuous, real-time, substantially no downtime data protection service to one or more data sources associated with a set of application host servers. To facilitate the data protection service, a host driver embedded in an application server captures real-time data transactions, preferably in the form of an event journal that is provided to other DMS components. The driver functions to translate traditional file/database/block I/O and the like into a continuous, application-aware, output data stream. The host driver includes an event processor. When a data protection command for a given data source is forwarded to a host driver, the event processor enters into an initial upload state. During this state, the event processor gathers a list of data items of the data source to be protected and creates a data list. Then, the event processor moves the data (as an upload, preferably one data element at a time) to a DMS core to create initial baseline data. In an illustrative embodiment, the upload is a stream of granular application-aware data chunks that are attached to upload events. Simultaneously, while the baseline is uploading and as the application updates the data on the host, checkpoint granular data, metadata, and data events are continuously streamed into the DMS core. During this upload phase, the application does not have to be shutdown.