Cache Flush Compression Using Reusable Data Page Dictionaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies in achieving high levels of data compression when flushing data pages from cache to non-volatile storage, particularly due to the resource-intensive nature of dictionary-based compression methods.
Innovation Solution
The technology identifies a compression group of similar data pages within a flush set and uses a previously generated compression dictionary for a similar data page stored in non-volatile storage to compress the group into a single block, reducing the need to store the dictionary again and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If dictionary-based compression methods are used to compress data pages, then compression ratio is improved, but resource consumption increases
Solution Approach 1:
The patent merges multiple similar data pages into a single compressed block by identifying pages with similar content through hash comparison. Instead of compressing each page separately with full dictionary generation, the system combines them into one compression operation, reducing redundant resource consumption while maintaining high compression ratios for the merged data.
Solution Approach 2:
The patent performs preliminary actions by pre-computing hash values for data pages and organizing them in hash indexes before the actual compression process. This preliminary organization allows the system to quickly identify similar pages and prepare compression groups in advance, reducing the computational overhead during the actual compression operation.
2Quantity of substance
If compression dictionary is generated for each data page, then compression effectiveness is improved, but processing time increases
Solution Approach 1:
The patent combines multiple data pages into compression groups and generates a single compression dictionary for the entire group rather than separate dictionaries for each page. This merging approach maintains compression effectiveness by capturing common patterns across similar pages while significantly reducing the total processing time by eliminating redundant dictionary generation operations.
3Adaptability or versatility
If multiple compression dictionaries are stored, then compression capability is improved, but storage space consumption increases
Solution Approach 1:
The patent merges the storage requirement for multiple compression dictionaries into a single dictionary stored once per compression group. By identifying similar pages through hash comparison and treating them as a group, the system stores only one dictionary per group rather than separate dictionaries for each page, thereby maintaining compression capability across all similar pages while reducing overall storage space consumption.
Data Source
AI summary
A flush set of data pages to flush from a cache to non-volatile data storage is identified, and a compression group is identified that is made up of similar data pages within the flush set. A previously compressed data page is identified in non-volatile data storage that is similar to the data pages in the compression group. Data compression is performed on the compression group using a compression dictionary for the previously compressed data page. A flush set specific hash index and a global hash index may be used to identify the similar data pages in the compression group and the similar previously compressed data page. An indication of the similar previously compressed data page may be stored in the compressed block generated for the compression group, and the compression dictionary for the previously compressed data page need not be duplicated in the non-volatile data storage.