Data Storage System Managing Compressed Data Block Deletion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in efficiently managing the deletion of individual data units within compressed data blocks, particularly in maintaining compatibility with read operations and ensuring that pointers are correctly updated, while also complying with privacy laws that require complete data removal.
Innovation Solution
A system and method for managing storage of data units that includes storing multiple compressed data blocks with historical information about prior removals, allowing for the replacement of deleted data units with new blocks and adjusting offsets based on this information, ensuring that read operations can function correctly even after deletions without updating all pointers immediately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data units are deleted from compressed data blocks by replacing blocks, then complete data removal is achieved, but pointer address information becomes invalid and requires correction
Solution Approach 1:
The patent stores historical information about deleted data units in advance, including their offsets and sizes, so that when read operations need to access data, the system can proactively adjust pointers using this pre-stored information rather than recalculating addresses on-demand
Solution Approach 2:
The patent introduces historical information as an intermediary data structure that mediates between the deleted data units and the read operations. This historical information contains offset and size data that allows the system to correct pointers without directly accessing the deleted data or reconstructing the entire data block
2Measurement precision
If all pointers are updated immediately after deletion, then address information remains accurate, but system complexity and processing time increase
Solution Approach 1:
The patent performs preliminary storage of historical information about deleted data units at the time of deletion, but delays the actual pointer correction until it is needed for read operations. This allows the system to prepare correction data in advance without immediately applying it to all pointers
Solution Approach 2:
The patent makes the pointer correction process dynamic by adjusting pointers on-demand based on whether they are actually needed for read operations. Not all pointers are corrected immediately, but only those that will be used, making the system adaptive to actual access patterns
3Reliability
If compressed data blocks are replaced after deletion, then data integrity is maintained, but the storage structure becomes more complex
Solution Approach 1:
The patent nests historical information within the compressed data block structure itself, storing offset and size information about deleted data units as part of the block's metadata. This nested structure allows the system to maintain data integrity while keeping the complexity contained within individual blocks rather than requiring complex external management structures
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A system for managing storage of data units includes a data storage system (106) configured to store multiple data blocks (202A-202C), at least some of the data blocks containing multiple data units, and configured to store, for at least some of the data blocks, corresponding historical information (214) about prior removal of one or more data units from that data block, the removal affecting at least some addresses of data units in that data block. The system is configured to perform at least one operation that accesses at least a first data unit stored in a first data block according to address information interpreted based on any stored historical information corresponding to the first data block.