Database Backup via Continuous Change Log Accumulation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current database backup solutions are inefficient, taking too long to perform backups, interrupting regular workflows, and providing limited options for restoring data to a specific point-in-time, often resulting in data loss and non-compliance with retention requirements.

Innovation Solution

A continuous data protection system that generates backups of database tables as of a past time by accumulating changes and using snapshots and change logs to create a consistent backup across partitions, allowing for on-demand or scheduled backups without interrupting production workloads, and enabling restoration to any point-in-time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If scan-based backup solutions are used to backup large databases, then backup completeness is improved, but backup time and resource consumption increase significantly

Engineering Contradiction:

Improvebackup completenessVSAvoidbackup time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by continuously capturing and storing change logs and transaction information in the background during normal database operations. This preliminary data collection enables rapid backup generation without requiring full database scans at backup time, thus completing backups quickly while ensuring completeness through the accumulated change data.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The backup system segments the database into multiple partitions and processes them independently. Change logs and transaction data are captured and stored separately for each partition, allowing parallel processing during backup operations. This segmentation reduces the time required to backup large databases while maintaining completeness through systematic coverage of all partitions.

Inventive Principle:

Principle #1Segmentation

2Productivity

If periodic backup systems are used, then resource competition is reduced, but restore flexibility to specific point-in-time is lost

Engineering Contradiction:

Improveresource availabilityVSAvoidrestore flexibility

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The system maintains continuous data protection by continuously capturing and storing change logs and transaction information in the background during normal database operations. This continuous action ensures that backup data is always available without interrupting production workloads, providing both resource availability and flexible restore capabilities to any point-in-time.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The backup system dynamically adapts to different restore requirements by allowing restoration to any specific point-in-time based on the continuously captured change logs. The system can dynamically generate backups at any moment without following a fixed periodic schedule, providing both resource efficiency and restore flexibility simultaneously.

Inventive Principle:

Principle #15Dynamics

3Measurement precision

If full database scans are performed for backups, then backup accuracy is improved, but production workload performance deteriorates

Engineering Contradiction:

Improvebackup accuracyVSAvoidproduction workload performance

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system extracts the backup function from the production database by capturing change logs and transaction information separately in the background. This extraction allows backup operations to proceed independently without scanning or blocking production workloads, maintaining both backup accuracy through complete change capture and production performance through non-intrusive operation.

Inventive Principle:

Principle #2Taking out (Extraction)

4Stability of the object's composition

If backup systems require stopping all writes for consistency, then backup consistency is improved, but workflow interruption increases

Engineering Contradiction:

Improvebackup consistencyVSAvoidworkflow continuity

Core Design Contradiction:

Stability of the object's compositionVSEase of operation

Solution Approach 1:

The system introduces change logs and transaction information as intermediary data structures that capture database changes without requiring write operations to stop. These intermediaries provide a consistent view of database state at any point-in-time while allowing production writes to continue uninterrupted, thus maintaining both backup consistency and workflow continuity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12013764B2Past-state backup generator and interface for database systems

Publication Date: 2024.06.18 AMAZON TECH INC

AI summary

An interface for requesting, and technique for generation of, a backup of a past state of a database table are provided. Changes made to a database table are accumulated, in durable storage, and snapshots of partitions of the table are obtained. The accumulated changes and the successive partition snapshots are used to generate a past state of the database at any point in time across a continuum between successive snapshots. Although each partition of the table may have a snapshot that was generated at a time different from when other partition snapshots were generated, changes from respective change logs may be selectively log-applied to distinct partitions of a table to generate backup in the past of the entire table at common point-in-time across partitions.