Similarity-Based Data Compression for Lower Read Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Previous data storage technologies compress large amounts of dissimilar data together, leading to increased overhead in read operations and reduced performance due to the need to decompress larger data sets, without significant improvements in compression ratios.
Innovation Solution
The technology selectively compresses data by generating hash values for candidate pages, identifying and grouping similar pages together for separate compression, thereby avoiding the compression of dissimilar data and reducing read overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large amounts of dissimilar data are compressed together, then compression ratio is improved, but read operation overhead increases and performance deteriorates
Solution Approach 1:
The patent segments the data compression process by dividing candidate pages into multiple groups based on similarity metrics. Instead of compressing all data uniformly, the system creates separate compression streams for similar pages, allowing selective decompression during read operations and improving overall system performance.
Solution Approach 2:
The patent applies local quality by treating different data pages with different compression strategies based on their similarity characteristics. Pages with high similarity are grouped together for compression, while dissimilar pages are handled separately, optimizing both compression efficiency and read performance for each group.
2Volume of stationary object
If dissimilar data pages are compressed together, then storage space is reduced, but decompression overhead increases
Solution Approach 1:
The patent segments pages into similarity-based groups before compression, creating multiple smaller compressed units rather than one large compressed block. This segmentation enables selective decompression of only the necessary pages during read operations, reducing decompression overhead while maintaining storage efficiency.
Solution Approach 2:
The patent performs preliminary similarity assessment and grouping of pages before compression. By pre-organizing pages into similarity-based groups, the system prepares the data structure to enable faster, more selective decompression operations later, avoiding the need to decompress entire large blocks when only small portions are needed.
3Measurement precision
If hash value generation and page similarity comparison is performed, then selective compression accuracy is improved, but processing overhead increases
Solution Approach 1:
The patent introduces hash values as an intermediary mechanism to efficiently compare page similarity. Instead of directly comparing entire pages which would be computationally expensive, the system uses hash values as a lightweight proxy to identify similar pages, significantly reducing processing overhead while maintaining acceptable accuracy.
Solution Approach 2:
The patent creates simplified representations (hash values) of the actual page data that can be quickly compared. These hash copies serve as efficient proxies for similarity detection, allowing the system to identify similar pages without the computational burden of full page comparisons.
Data Source
AI summary
Technology is disclosed for selectively compressing data based on similarity of pages within the data that is to be compressed. At least one corresponding hash value is generated for each one of multiple candidate pages to be compressed. In response to the hash values generated for the candidate pages, the technology selects a set of similar candidate pages from the candidate pages. The set of similar candidate pages are a subset of the candidate pages that includes less than all the candidate pages. The set of similar candidate pages are compressed as a single unit, separately from one or more other ones of the candidate pages that were not selected to be included in the set of similar candidate pages.