Log-Based Storage Data Consolidation via Temporal Order Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional log-structured data storage systems inefficiently mix unrelated data, leading to increased access time, fragmentation, and defragmentation efforts due to proactive data combination in new segments, whereas temporally-related data is not effectively consolidated.
Innovation Solution
A method involving a garbage collection service that updates a data order log to track temporal order, consolidates data blocks received around the same time, and reorganizes them in secondary storage to maintain temporal relationships, optimizing storage efficiency and reducing fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the log-structured data storage system proactively combines valid data from partially-used segments and stores the combined valid data in new unoccupied segments, then storage space is freed for reuse, but data that has little or no relation is mixed together leading to increased access time and fragmentation
Solution Approach 1:
The patent applies preliminary action by maintaining a data order log that tracks the temporal order of data blocks when they are initially written to storage. This preliminary tracking enables subsequent consolidation operations to efficiently retrieve and reorganize data blocks in their original temporal sequence, avoiding the need to mix unrelated data while still freeing storage space.
Solution Approach 2:
The patent changes the organization parameter of data consolidation from arbitrary combination to temporal-order-based combination. By using the data order log to track and enforce temporal relationships during consolidation, the system transforms how data is reorganized in storage, ensuring that related data blocks remain together while still achieving space reuse.
2Extent of automation
If the log-structured data storage system proactively combines valid data from partially-used segments, then storage management is automated, but subsequent fragmentation increases requiring more defragmentation work
Solution Approach 1:
The system performs preliminary action by recording the temporal order of data blocks in a data order log at the time of initial writing. This preliminary record enables automated consolidation operations to automatically reconstruct the original data sequence without requiring complex defragmentation algorithms, thereby maintaining automation while reducing subsequent complexity.
Solution Approach 2:
The data order log serves as an intermediary structure that mediates between the automated consolidation process and the physical data arrangement. This intermediary tracks temporal relationships and guides the consolidation operation to place data blocks in their correct temporal sequence, eliminating the need for complex post-consolidation defragmentation.
3Productivity
If data blocks are scattered across different segments without temporal consideration, then storage space utilization is maximized, but sequential reads and prefetching efficiency deteriorate
Solution Approach 1:
The patent changes the data organization parameter from space-optimization-only to a dual consideration of both space utilization and temporal locality. By incorporating temporal order information from the data order log into the consolidation process, the system arranges data blocks to maintain their temporal relationships, enabling efficient sequential reads and prefetching while still achieving high storage utilization through proactive consolidation.
Data Source
AI summary
Technique manage data within computerized memory. The techniques involve, in response to receiving host data in a write cache, updating a data order log that holds order information indicating a temporal order for the host data. The temporal order initially is the order that the host data was received in the write cache. The techniques further involve transferring the host data from the write cache to secondary storage. The techniques further involve, after the host data is transferred from the write cache to secondary storage, providing a garbage collection service that consolidates the host data within the secondary storage in accordance with the data order log that holds the order information indicating the temporal order for the host data. With the temporal order of the host data generally preserved, data access operations may enjoy various optimizations such as improved prefetching, more sequential reads, improved auto-tiering, and so on.