Inline Data Compression Fragmentation Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face inefficiencies in managing inline data compression, leading to fragmentation, wastage of storage space, and reduced performance due to frequent allocation of new data fragments for updates, which results in performance bottlenecks and increased flash wear.
Innovation Solution
Implementing a method to manage and track free data fragments for re-use in storage systems, allowing updated data to be written in the existing compressed format within existing data fragments, thereby reducing the need for new allocations and minimizing fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If new data fragments are allocated for every update operation, then data can be stored in compressed format, but storage space is wasted and fragmentation increases
Solution Approach 1:
The patent implements a free list data structure that tracks and recovers data fragments that have been freed from the file system. When data is deleted or a file is truncated, the corresponding data fragments are added to the free list. During allocation operations, the system first checks the free list for suitable fragments before allocating new space, thereby recovering and reusing previously freed storage space instead of wasting it.
Solution Approach 2:
The system performs preliminary organization of free data fragments by maintaining a sorted free list where fragments are ordered by size and location. This preliminary arrangement allows the allocation algorithm to efficiently find suitable fragments for reuse before the actual allocation occurs, preventing the need to scan the entire storage space and reducing fragmentation.
2Reliability
If new allocation units are created for updated data, then compression can be maintained, but IO performance decreases due to frequent allocations
Solution Approach 1:
The patent recovers data fragments from deleted files and adds them to a free list for reuse. When updating data, the system allocates from this pre-prepared free list rather than creating new allocation units, significantly reducing the overhead of allocation operations and improving IO performance while maintaining compression integrity.
Solution Approach 2:
The free list data structure serves itself by automatically tracking freed fragments and making them available for future allocations. This self-service mechanism eliminates the need for complex allocation algorithms to scan and identify reusable space, reducing CPU overhead and improving allocation speed.
3Adaptability or versatility
If continuous new allocations are performed, then updated data can be stored, but flash wear increases
Solution Approach 1:
The patent recovers and reuses data fragments from deleted files through the free list mechanism. By allocating from existing freed space rather than continuously creating new allocations, the system reduces the number of write amplification cycles on flash storage, thereby extending flash memory lifespan while maintaining full data update capability.
4Reliability
If data fragments are frequently allocated and created, then updated data can be stored in compressed format, but storage fragmentation increases
Solution Approach 1:
The system performs preliminary organization of free data fragments by maintaining a sorted free list where fragments are ordered by size and location. This preliminary arrangement allows the allocation algorithm to efficiently find suitable fragments for reuse, reducing random allocations that cause fragmentation and maintaining storage structure stability.
Solution Approach 2:
By recovering and reusing freed data fragments through the free list, the system reduces the creation of new scattered allocation units. Reusing existing fragments maintains the原有的 storage structure and reduces fragmentation compared to continuously creating new allocation units.
Data Source
AI summary
A method is used in managing inline data compression in storage systems. A request is received to update data of a data object previously stored in an allocation unit of a segment in a storage system. The segment comprises of a set of allocation units in a compressed format. A determination is made as to whether the updated data of the data object can be stored in the compressed format in the allocation unit of the segment. Based on the determination, a free allocation unit is selected from a list of free allocation units managed by the storage system for writing the updated data in the compressed format to the free allocation unit.