Compressed Data Overwrites Without New Mapping Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inline data compression in data storage systems can lead to over-commitment of storage resources and increased disk traffic due to the need for new space allocation during overwrites, which slows down processing and decreases storage efficiency.
Innovation Solution
A method that tests whether new data compressed with a first compression procedure fits within existing space; if it doesn't, a second, more aggressive compression procedure is used to reduce the size, allowing the data to be stored in the same location without creating new metadata structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If new data is compressed using a first compression procedure and stored in the space occupied by previous data, then storage efficiency is improved, but if the compressed new data does not fit, new space must be allocated which decreases storage efficiency and slows down processing
Solution Approach 1:
The patent applies parameter changes by switching between different compression procedures (parameters) based on the fit test result. When the first compression procedure does not produce data that fits in the allocated space, the system changes to a second compression procedure with different parameters to achieve better compression ratios, thereby resolving the contradiction between maintaining storage efficiency and accommodating variable data sizes.
2Reliability
If new space is allocated to accommodate compressed data that does not fit in existing space, then data storage is ensured, but new mapping metadata must be created which decreases storage efficiency
Solution Approach 1:
The patent applies preliminary action by performing a fit test before finalizing the storage location. This preliminary check determines whether the compressed data will fit in the originally allocated space, allowing the system to proactively adjust compression parameters or allocate different space before the write operation completes, thereby avoiding the need to create new mapping metadata later.
3Productivity
If background compression is performed on already-stored data, then storage efficiency is improved, but disk traffic increases significantly
Solution Approach 1:
The patent applies preliminary action by performing compression at the time of data ingestion rather than as a background operation on already-stored data. This allows the system to compress data before it is written to disk, eliminating the need for subsequent background compression operations and the associated increase in disk traffic.
4Productivity
If inline compression is performed on incoming data, then over-commitment of storage resources is avoided, but new compressed data may not fit in the space provided for original data
Solution Approach 1:
The patent applies dynamics by making the compression process adaptive rather than static. The system dynamically selects between different compression procedures based on the characteristics of the incoming data and the available space, allowing it to adapt to varying data sizes and compression requirements while maintaining efficient storage resource utilization.
Data Source
AI summary
A technique for overwriting compressed data tests whether new data compressed with a first compression procedure fits within spaces provided for previous data. If the compressed new data does not fit, the technique compresses the new data using a second compression procedure. Assuming the second compression procedure reduces the compressed size of the new data to fit the available space, the technique stores the new data in the same location as the previous data. In this manner, overwrites can be accommodated in place without the need to create new mapping metadata.