Snapshot Difference Buffer with Checkpoint Resumption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data management systems face challenges in efficiently updating remote storage by sending incremental changes, especially when network issues or crashes occur, leading to data integrity and redundancy problems.
Innovation Solution
A data management device that calculates and sends incremental changes between local and previous snapshots, filters these changes, and uses cookies to track applied changes, allowing for resumed updates after crashes, thereby ensuring data consistency and reducing redundant data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If incremental changes are sent to update remote storage, then data redundancy is improved, but data integrity may deteriorate due to network issues or crashes
Solution Approach 1:
The patent applies preliminary action by calculating incremental changes between local and previous snapshots before transmission, and by implementing a checkpoint mechanism that records the state before updates. This allows the system to resume from known good states if crashes occur, preventing data integrity issues while maintaining redundancy benefits.
Solution Approach 2:
The patent implements feedback through a checkpoint system that tracks which incremental changes have been successfully applied to remote storage. The system continuously monitors update status and can rollback or resume operations based on feedback from the remote storage system, ensuring data integrity is maintained even during network issues or crashes.
2Reliability
If all changes are re-sent after a crash, then data integrity is restored, but transmission time increases
Solution Approach 1:
The patent applies preliminary action by maintaining checkpoints that record the state of remote storage before updates begin. After a crash, the system uses these pre-established checkpoints to determine exactly where to resume, avoiding the need to re-transmit already-applied changes and significantly reducing recovery time.
Solution Approach 2:
The patent implements discarding and recovering by identifying and discarding (skipping) incremental changes that were already successfully applied before the crash, while recovering only the changes that were lost or incomplete. This selective approach minimizes re-transmission time while ensuring data integrity.
3Productivity
If incremental updates are implemented, then transmission efficiency is improved, but complexity of tracking applied changes increases
Solution Approach 1:
The patent applies segmentation by dividing the update process into discrete incremental changes, each associated with a specific checkpoint. This segmentation allows the system to track and manage individual changes independently, making the tracking process more manageable despite the complexity of incremental updates.
Solution Approach 2:
The patent introduces an intermediary checkpoint mechanism that mediates between the local snapshot data and remote storage. This intermediary layer handles the complexity of tracking which changes have been applied by providing a standardized interface and state recording system, simplifying the overall tracking process.
4Loss of energy
If snapshots are compared to generate incremental changes, then data transmission is optimized, but computational resources increase
Solution Approach 1:
The patent applies partial action by comparing only the portions of snapshots that have changed since the previous update, rather than comparing entire snapshots. This selective comparison reduces computational resources required while still achieving optimized data transmission by sending only the necessary incremental changes.
Data Source
AI summary
A data management device includes a buffer and a processor. The processor may select an unprocessed full key and generate a buffer entry based on a difference between an entry of a local snapshot specified by the unprocessed full key and an entry of a previous local snapshot specified by the unprocessed full key. The processor may make a first determination that a lookup key entry associated with the unprocessed full key is different than a lookup key entry associated with a processed full key. The process may add the generated buffer entry to the buffer after processing the buffer in response to the first determination.