Database Sync Change Log Compaction for Efficient Synchronization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current database synchronization methods are inefficient as they require scanning the entire database for changes, leading to increased computational overhead and inefficiency, especially with large datasets and frequent updates.

Innovation Solution

A change log compaction data structure is generated, categorizing changes into exact, range, and interval entries, allowing for efficient querying and synchronization by focusing on specific data ranges and time intervals, reducing the need to scan the entire database.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the entire database is scanned for changes, then all changes are detected, but computational overhead increases and synchronization efficiency decreases

Engineering Contradiction:

Improvechange detection completenessVSAvoidsynchronization efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The database change log is segmented into discrete change indicators, each representing a specific modification event. These change indicators are organized into a compaction data structure that divides the change tracking task into manageable units, allowing efficient querying without scanning the entire database. Each change indicator contains specific metadata about the change type, version, and affected data, enabling targeted synchronization.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A change log compaction data structure is introduced as an intermediary between the raw database changes and the synchronization process. This intermediate structure pre-processes and organizes change information into a query-optimized format, acting as a mediator that translates comprehensive change tracking into efficient selective retrieval operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If detailed change information is tracked for all database modifications, then synchronization accuracy is improved, but data transfer volume and storage requirements increase

Engineering Contradiction:

Improvesynchronization accuracyVSAvoiddata transfer volume

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

Only the essential change information is extracted and stored in the change log compaction data structure. Each change indicator contains only the necessary metadata (change type, version, affected data identifiers) rather than full data copies. This extraction approach maintains synchronization accuracy by tracking all changes while minimizing data transfer volume by transmitting only change metadata.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of transmitting full data sets and applying changes, the system inverts the approach by transmitting only change indicators that describe what changed. The client then applies these changes locally based on the compact change metadata, reducing data transfer volume while maintaining synchronization accuracy.

Inventive Principle:

Principle #13The other way round (Inversion)

3Speed

If change logs are maintained for frequent updates, then real-time synchronization is enabled, but computational overhead and processing time increase

Engineering Contradiction:

Improvesynchronization speedVSAvoidcomputational overhead

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

Change indicators are pre-processed and organized into the compaction data structure as changes occur, rather than processing all changes at synchronization time. This preliminary organization creates an optimized query structure that enables fast retrieval during synchronization, reducing both processing time and computational overhead when actual sync operations occur.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system replaces brute-force scanning mechanics with optimized data structure queries. Instead of mechanically scanning through all database changes sequentially, the compaction data structure enables direct access to relevant changes through indexed queries, significantly reducing computational overhead and processing time for frequent updates.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12164481B2Database syncing

Publication Date: 2024.12.10 GOOGLE LLC

AI summary

A method for database syncing is provided. The method includes receiving database change indicators indicating changes to a database. For each database change indicator, the method also includes determining whether the respective change to the database corresponds to a single change to a single version of the database, one of multiple changes to a common version of the database, or one of multiple changes to different versions of the database. The method further includes generating a change log compaction data structure comprising entries. Each entry includes an exact entry defining a corresponding single change to a corresponding single version of the database, a range entry defining corresponding multiple changes to a corresponding common version of the database, or an interval entry defining corresponding multiple changes to corresponding different versions of the database.