Storage Data Compression via Reinforcement Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data compression systems in storage devices face challenges in optimizing the tradeoff between processing resources and compression efficiency, as the type of data and access frequency can vary over time, making it difficult to select a single optimal compression algorithm.

Innovation Solution

The system dynamically selects portions of data unlikely to be accessed and applies specific compression algorithms based on frequency values, adjusting these values according to the performance ratio of compression to processing cycles, using both hardware and software compression methods, including out-of-band and in-line compression.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If greater amounts of compression are applied to data, then the storage space efficiency is improved, but the processing resources consumed increase

Engineering Contradiction:

Improvestorage space efficiencyVSAvoidprocessing resources

Core Design Contradiction:

Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

The system applies different compression algorithms to different data portions based on their access frequency and characteristics. Frequently accessed data uses lighter compression while less frequently accessed data uses more aggressive compression, optimizing the local tradeoff between storage efficiency and processing resources for each data segment.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The compression algorithm selection is dynamically adjusted based on changing data access patterns over time. The system monitors access frequency and adapts the compression strategy accordingly, transitioning from static to dynamic algorithm selection to optimize the compression-resource tradeoff as data usage evolves.

Inventive Principle:

Principle #15Dynamics

2Quantity of substance

If compression algorithms are applied to frequently accessed data, then the storage space efficiency is improved, but the processing time increases

Engineering Contradiction:

Improvestorage space efficiencyVSAvoidprocessing time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system identifies frequently accessed data portions and applies lighter or no compression to these segments, while applying more aggressive compression to less frequently accessed data. This local differentiation reduces the processing time penalty for compression while maintaining storage efficiency benefits on appropriate data segments.

Inventive Principle:

Principle #3Local quality

3Device complexity

If a single compression algorithm is selected for all data, then the system complexity is reduced, but the adaptability to different data types and access patterns decreases

Engineering Contradiction:

Improvesystem complexityVSAvoidadaptability to data types

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system segments data into different portions based on access frequency and characteristics, then selects different compression algorithms for each segment. This segmentation approach enables adaptability to different data types and access patterns while keeping the complexity of each individual algorithm selection manageable.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements a universal compression framework that can select from multiple compression algorithms based on data characteristics. This multi-functional approach allows a single system to handle diverse data types and access patterns effectively, achieving high adaptability without requiring separate dedicated systems for each data type.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Quantity of substance

If compression is applied to all data regardless of access frequency, then the storage space efficiency is improved, but the processing overhead increases unnecessarily

Engineering Contradiction:

Improvestorage space efficiencyVSAvoidprocessing overhead

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The system applies compression selectively to data portions based on their access frequency characteristics. Frequently accessed data may receive lighter or no compression, while less frequently accessed data receives more aggressive compression. This local differentiation reduces unnecessary processing overhead while maintaining storage efficiency benefits where they are most valuable.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS10466921B1Accelerating data reduction through reinforcement learning

Publication Date: 2019.11.05 EMC IP HLDG CO LLC

AI summary

Compressing data of a storage device includes selecting a portion of data for data compression that is predicted to be unlikely to be accessed, selecting a particular one of a plurality of data compression algorithms to apply to the portion according to a frequency value associated with each of the data compression algorithms, and adjusting the frequency value of the particular one of a plurality of data compression algorithms according to performance of the particular one of a plurality of data compression algorithms. The performance may vary according to a ratio of amount of compression achieved to processing cycles of a processor used to compress the portion of data. The processor may perform storage device functions that are separate from data compression. The portion of data may be selected from a plurality of logical devices used in connection with a single application that accesses the storage device.