Multiple Stream Data Compression for Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face challenges in efficiently storing and retrieving variable-sized compressed data, which requires significant changes to data storage and retrieval mechanisms, and complicates the storage of interleaved write commands due to compressed data spanning multiple physical locations.
Innovation Solution
The implementation of a data storage system that compresses and formats host data into fixed-size virtual storage units, including error correction and metadata, allowing for de-interleaving and formatting of multiple write streams, and utilizing a compression and formatting module to manage data in a way that sequential data is stored adjacent in physical memory locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed before storage, then storage capacity is increased, but data retrieval mechanisms become significantly more complex and variable-sized data must be stored
Solution Approach 1:
The patent segments compressed data into fixed-size containers called virtual storage units (VSUs). Each VSU has a predetermined size and contains a header portion with metadata and a data portion with compressed data. This segmentation allows variable-sized compressed data to be organized into uniform units, simplifying storage and retrieval mechanisms while maintaining compression benefits.
Solution Approach 2:
The patent introduces virtual storage units as an intermediary layer between the compressed data and the physical storage medium. These VSUs act as mediators that translate variable-sized compressed data into fixed-size storage units, enabling standard storage mechanisms to handle compressed data without requiring complex variable-size handling logic.
2Productivity
If multiple interleaved write streams are processed, then data throughput is improved, but de-interleaving and formatting becomes more difficult when compression is applied
Solution Approach 1:
The patent segments multiple interleaved write streams into separate virtual storage unit streams. Each write stream is processed independently to create its own sequence of VSUs, which can then be de-interleaved and formatted more easily. The fixed-size nature of VSUs makes them easier to manage and reorganize compared to variable-sized compressed data blocks.
Solution Approach 2:
The patent performs preliminary compression and VSU formation on each write stream before the de-interleaving process. By pre-processing each stream into standardized VSUs with embedded metadata, the system simplifies subsequent de-interleaving and formatting operations, as the heavy lifting of compression and organization is already complete.
3Quantity of substance
If variable-sized compressed data is stored directly, then storage efficiency is improved, but sequential data cannot be stored adjacent in physical memory locations
Solution Approach 1:
The patent segments compressed data into fixed-size virtual storage units, each with a predetermined size. This segmentation enables sequential data to be stored adjacent in physical memory locations, as each VSU occupies a uniform space. The fixed-size containers maintain storage efficiency through compression while enabling regular memory arrangement for better performance.
Solution Approach 2:
The patent changes the size parameter of compressed data from variable to fixed by encapsulating variable-sized compressed content within fixed-size virtual storage units. This parameter change enables sequential data to be stored adjacently in physical memory, improving access patterns and performance while maintaining the space-saving benefits of compression.
Data Source
AI summary
Embodiments of multiple stream compression and formatting of data for data storage systems are disclosed. In some embodiments, a data storage system can compress multiple streams of fixed sized host data before storing it on a media and format obtained variable sized compressed data for storing on the media that typically has fixed size storage granularity. One or more modules compress the incoming host data and create multiple output streams of fixed sized storage units that contain compressed data. The storage units are stored on the media. Capacity, reliability, and performance are thereby increased.