Data Chunk Pattern Replacement for Storage Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data compression in storage systems is computationally expensive, necessitating alternative methods to reduce data size efficiently.
Innovation Solution
A technique that identifies and replaces contiguous regions of repeating data within a chunk with a single instance, reducing the chunk's size and allowing for further compression at lower computational cost, thereby reducing processor, memory, and bus access demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is performed to reduce storage space, then storage efficiency is improved, but computational cost increases
Solution Approach 1:
The patent applies preliminary action by performing data reduction through pattern replacement before compression. The system identifies repeating patterns and replaces them with references to single instances, creating a pre-processed data structure that requires less computational effort during subsequent compression operations. This preliminary structuring of data reduces the overall computational burden while achieving both space and time efficiency.
2Quantity of substance
If compression is performed on full-size chunks, then storage efficiency is improved, but processing time increases
Solution Approach 1:
The patent applies segmentation by dividing data into chunks and further segmenting each chunk into manageable units for pattern identification and replacement. By organizing data into structured segments with clear boundaries and metadata, the system enables efficient processing of large datasets without requiring excessive processing time, as each segment can be independently analyzed and optimized.
3Quantity of substance
If repeating regions are replaced with single instances, then data size is reduced, but memory and computational resources are consumed during identification
Solution Approach 1:
The patent applies copying by creating simplified representations of repeating patterns rather than storing actual copies of the repeated data. The system identifies patterns and replaces multiple instances with references to a single stored instance, using metadata to track pattern locations and lengths. This approach reduces data size while consuming minimal memory and computational resources during the identification and replacement process.
Data Source
AI summary
A technique for managing data storage begins at a predetermined offset relative to a chunk of data received for writing, and identifies a span of contiguous regions of the chunk that contain identical data. The technique replaces the span of contiguous regions of the chunk with a single instance of a region of the contiguous regions. The technique persistently stores a shortened version of the chunk with the single instance replacing the span of contiguous regions.