Real-time Data Compression Domain Classification via Fingerprinting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data compression methods in computer systems are inefficient as they lack real-time classification of data into appropriate compression domains, leading to degraded compression ratios due to mixing dissimilar data types, such as images and text, within the same compression stream.

Innovation Solution

A method that uses a processor to read randomly selected data from write operations, compute classifying heuristics, and create fingerprints to determine which data compression domains to forward write operations, allowing similar data to be compressed together in the same stream, thereby enhancing compression efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Area of moving object

If data is compressed without real-time classification into compression domains, then the compression process is simple and fast, but the compression ratio is degraded due to mixing dissimilar data types

Engineering Contradiction:

Improvecompression ratioVSAvoidclassification system complexity

Core Design Contradiction:

Area of moving objectVSDevice complexity

Solution Approach 1:

The patent segments data into different compression domains (image, text, mixed, etc.) based on fingerprint classification. Each domain is then compressed using domain-specific compression algorithms, replacing the single unified compression approach. This segmentation enables higher compression ratios for each data type while maintaining manageable system complexity through modular domain handlers.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system changes the parameter of data representation by creating fingerprints that capture essential characteristics of data without full compression. These fingerprints serve as classification parameters that determine the appropriate compression domain, enabling efficient routing without requiring complete data analysis.

Inventive Principle:

Principle #35Parameter changes

2Speed

If random data sampling is used for fingerprint creation, then classification speed is fast, but classification precision may be insufficient

Engineering Contradiction:

Improveclassification speedVSAvoiddata type classification accuracy

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The patent applies partial action by sampling only a subset of data (e.g., every nth byte or specific portions) to create fingerprints instead of analyzing entire data streams. This partial analysis achieves sufficient classification accuracy for most cases while maintaining high speed, with the option to perform deeper analysis only when needed based on initial fingerprint results.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS9792350B2Real-time classification of data into data compression domains

Publication Date: 2017.10.17 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

For real-time classification of data into data compression domains, a decision is made for which of the data compression domains write operations should be forwarded by reading randomly selected data of the write operations for computing a set of classifying heuristics thereby creating a fingerprint for each of the write operations. The write operations having a similar fingerprint are compressed together in a similar compression stream.