Dataset Storage Location Selection Using Compressibility Hints

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data storage systems face inefficiencies in selecting between compressed and non-compressed storage locations due to insufficient information, leading to suboptimal choices that waste computing resources and time.

Innovation Solution

Incorporating hinting information generated by applications or clients about dataset properties, such as compressibility and usage patterns, to guide data storage systems in selecting appropriate storage locations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the data storage system makes storage location decisions without hinting information, then the system operates with simpler data processing, but storage efficiency deteriorates due to suboptimal choices

Engineering Contradiction:

Improvestorage efficiencyVSAvoiddata processing complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The application or client generates hinting information about dataset properties (compressibility, usage patterns) before the data storage system makes its decision. This preliminary action provides the storage system with pre-analyzed characteristics, enabling more efficient storage location selection without requiring complex real-time analysis of the entire dataset.

Inventive Principle:

Principle #10Preliminary action

2Productivity

If the data storage system analyzes dataset properties without hinting information, then storage decisions can be optimized, but computing resources and time are wasted

Engineering Contradiction:

Improvestorage decision qualityVSAvoidtime for storage decision

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The hinting information extracts only the most relevant dataset properties (compressibility, usage patterns) that are needed for storage location decision-making. Instead of analyzing the entire dataset or all possible properties, the system extracts and processes only the essential characteristics, significantly reducing the time and computing resources required while maintaining decision quality.

Inventive Principle:

Principle #2Taking out (Extraction)

3Productivity

If hinting information is provided about dataset properties, then storage location selection is optimized, but additional data processing overhead is introduced

Engineering Contradiction:

Improvestorage location selection accuracyVSAvoidprocessing overhead

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system provides hinting information that includes more dataset properties than strictly minimal (such as both compressibility and usage patterns), which may be more than the absolute minimum needed. This partial excess ensures that the storage system has comprehensive information for making accurate decisions, and the additional processing overhead is offset by the improved storage location selection accuracy that results.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12405743B2Hinting information for selecting a storage location for a dataset

Publication Date: 2025.09.02 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Described are techniques for selecting compressed or non-compressed data storage using hints for a dataset. The techniques include receiving a dataset at a data storage system that manages storage location on compressed data storage and non-compressed data storage. The techniques further include determining, by the data storage system, that the dataset includes one or more hints for selecting a storage location for the dataset from one of the storage locations managed by the data storage system. The techniques further include selecting, by the data storage system, the storage location for the dataset based in part on the one or more hints and storing the dataset to the storage location.