Shared Update Log Data Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed data storage systems, recovering data sets from a failed server can be inefficient, leading to reduced system effectiveness due to the need for timely reassignment and reconstruction across multiple servers.
Innovation Solution
A method involving the recording of update records in a shared update log with non-overlapping sub-logs, where update records are reorganized into subsets and applied to corresponding data sets across multiple servers to reconstruct the data sets, ensuring efficient data recovery and redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a distributed system across multiple servers, then data access efficiency and system reliability are improved, but data recovery time increases when a server fails
Solution Approach 1:
The patent applies preliminary action by maintaining update logs that continuously record all data modifications before failures occur. When a server fails, the recovery process simply needs to apply these pre-recorded updates to baseline data, rather than reconstructing everything from scratch. This preliminary recording of update operations dramatically reduces recovery time while maintaining distributed system reliability.
Solution Approach 2:
The patent segments the update log into multiple sub-logs that can be processed independently and in parallel. Each sub-log contains a portion of the update records, allowing multiple servers to simultaneously work on different segments during recovery. This segmentation enables parallel processing of recovery operations, reducing overall recovery time while preserving the benefits of distributed storage.
2Stability of the object's composition
If update records are stored in a single centralized log, then data consistency is maintained, but recovery efficiency decreases due to bottlenecks
Solution Approach 1:
The patent divides the centralized update log into multiple independent sub-logs distributed across different servers. Each sub-log maintains a consistent portion of the update records, allowing parallel processing during recovery without creating bottlenecks. This segmentation preserves data consistency through proper indexing and ordering within each sub-log while dramatically improving recovery efficiency through parallel access.
Solution Approach 2:
The patent transitions from a single-dimensional centralized log structure to a multi-dimensional distributed sub-log structure. By organizing update records across multiple dimensions (different sub-logs on different servers), the system maintains consistency through logical ordering within each dimension while enabling parallel processing across multiple dimensions, thus improving recovery efficiency without sacrificing consistency.
3Manufacturing precision
If all update records are processed sequentially for each data set, then processing accuracy is maintained, but reconstruction time increases
Solution Approach 1:
The patent segments update records into subsets associated with specific data sets or groups of data sets. These segmented update subsets can be processed in parallel by different servers or processing units, each maintaining accuracy through proper application of updates to their assigned data sets. This segmentation enables concurrent processing while preserving processing accuracy through structured update application.
Solution Approach 2:
The patent applies partial action by processing only the relevant update records for each data set rather than all updates sequentially. Each server or processing unit focuses on a specific subset of updates applicable to its assigned data sets, applying them in the correct order to maintain accuracy. This partial processing approach significantly reduces reconstruction time by eliminating unnecessary sequential processing of unrelated updates.
Data Source
AI summary
Updates to a plurality of data sets may be recorded in a shared update log. Whenever a server responsible for the plurality of data sets fails, other servers may reconstruct the data sets using the shared update log and redundant copies of the data sets. As part of the reconstruction process, the other servers reorganize (e.g., by sorting) the update records of sub-logs of the update log to group update records corresponding to the same data set together. The updates are applied to the redundant copies of the data set snapshots to reconstruct the data set.