Multi-Tier Storage Data Placement and Request Reordering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud storage systems, particularly object storage, lack efficient mechanisms for optimizing data retrieval and movement across multi-tiered storage hierarchies, leading to suboptimal performance and increased latency due to lack of control over data placement and retrieval strategies.

Innovation Solution

A computer program product that manages storage in a multi-tiered system by receiving commands to determine optimal storage tiers for objects, optimizing data placement, and handling batch requests efficiently, allowing for collocation guidance to minimize seek times and improve data retrieval by moving data to faster storage tiers and reordering requests for optimal tape access.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data is stored in a multi-tiered storage hierarchy across flash, disk, and tape storage, then storage cost is reduced, but data retrieval latency increases when data resides on slower tiers

Engineering Contradiction:

Improvestorage capacity utilizationVSAvoiddata retrieval latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system performs preliminary actions by proactively moving data from slower storage tiers (tape) to faster tiers (disk, flash) based on predicted access patterns and application requirements. The storage manager analyzes command patterns and pre-positiones data on optimal tiers before actual access occurs, thereby reducing retrieval latency while maintaining cost-effective tiered storage architecture

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms by continuously monitoring data access patterns, command rates, and storage tier performance metrics. Based on this feedback, the storage manager dynamically adjusts data placement decisions, moving data between tiers to optimize the balance between storage cost and retrieval speed according to actual usage patterns

Inventive Principle:

Principle #23Feedback

2Device complexity

If object storage services hide the mapping of objects to storage tiers, then system complexity is reduced, but user ability to optimize data retrieval is lost

Engineering Contradiction:

Improvestorage management complexityVSAvoiddata retrieval efficiency

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The system enables self-service by providing applications with the tools and information needed to make their own optimization decisions. The storage manager exposes command rate information and access pattern data to applications, allowing them to self-optimize their data access strategies without requiring deep knowledge of storage tier mappings, thus maintaining simplicity while improving efficiency

Inventive Principle:

Principle #25Self-service

3Ease of operation

If multiple individual requests are handled separately for reading data blocks, then request processing simplicity is maintained, but seek time increases and performance deteriorates

Engineering Contradiction:

Improverequest handling simplicityVSAvoiddata read performance

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The system merges multiple individual data read requests into consolidated batch operations. The storage manager groups requests for data blocks that reside on the same storage tier or media, combining them into single optimized read operations. This reduces the number of separate seek operations required while maintaining ease of request submission, thereby improving data read performance

Inventive Principle:

Principle #5Merging (Combining)

4Quantity of substance

If data is distributed across multiple tapes for storage, then storage capacity is increased, but recall time increases when data must be retrieved from multiple tapes

Engineering Contradiction:

Improvestorage capacityVSAvoiddata recall time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system performs preliminary actions by consolidating data onto fewer tapes during the write phase when possible. The storage manager analyzes the data set and storage capacity requirements to determine optimal tape allocation, pre-positioning data in a manner that minimizes the number of tapes required for future recall operations, thus reducing recall time while maintaining adequate storage capacity

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system introduces a new dimension of optimization by considering temporal patterns in data access. Rather than simply distributing data across tapes based on capacity requirements, the storage manager analyzes access patterns and uses this temporal information to make intelligent decisions about data placement, grouping frequently accessed data in a manner that minimizes recall time

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS10740004B2Efficiently managing movement of large amounts object data in a storage hierarchy

Publication Date: 2020.08.11 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A computer program product is provided for efficiently managing storage in a multi-tiered storage system. The computer program product comprises a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor to cause the processor to receive a command from an application, where the command is directed to at least one object. The program instructions are further executable by the processor to cause the processor to determine storage for the at least one object in a multi-tiered storage system based on the command, and store the at least one object in accordance with the determined storage.