Probabilistic File Migration for Storage Policy Compliance

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing storage systems face challenges in efficiently managing files between high-speed and slower storage devices, as policy-based and capacity usage concepts often interfere with each other, leading to inefficiencies in file migration and potential out-of-space conditions.

Innovation Solution

A method that determines a policy-specific probability distribution for each file based on assigned attributes, allowing for a total probability calculation that dictates file migration from high-speed to slower storage devices, balancing policy-based and capacity usage concepts by moving files with a total probability of 1 independently and those with probabilities between 0 and 1 based on storage space thresholds.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If files are migrated based on policy criteria (age, size), then storage cost efficiency is improved, but storage capacity management becomes problematic when tier 1 storage is full and no files meet migration criteria

Engineering Contradiction:

Improvestorage cost efficiencyVSAvoidstorage capacity management flexibility

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent transforms rigid policy-based file migration into a probabilistic model where files are assigned migration probabilities based on multiple attributes (age, size, access patterns). This allows the system to adaptively select files for migration even when traditional policy criteria are not met, resolving the contradiction between cost efficiency and capacity management flexibility.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system dynamically adjusts file migration decisions by computing probabilities that can change over time based on file attributes and storage conditions. This dynamic approach allows the system to respond to changing storage capacity needs while maintaining cost efficiency through intelligent file selection.

Inventive Principle:

Principle #15Dynamics

2Manufacturing precision

If all files must meet policy criteria to be migrated, then policy compliance is maintained, but storage space management becomes inflexible when tier 1 storage reaches 100% capacity

Engineering Contradiction:

Improvepolicy compliance accuracyVSAvoidstorage space management ease

Core Design Contradiction:

Manufacturing precisionVSEase of operation

Solution Approach 1:

Instead of requiring 100% policy compliance for file migration, the system applies partial action by migrating files with high migration probabilities even if they don't strictly meet all policy criteria. This partial compliance approach prevents storage overflow while maintaining reasonable policy adherence.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The migration probability acts as an intermediary mechanism between strict policy compliance and storage space management. It translates policy attributes into probabilistic decisions, allowing the system to balance compliance requirements with practical storage management needs.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If a score-based policy is used to select files for migration, then storage capacity management is improved, but system administrators lose the ability to define exact age or size criteria for file migration

Engineering Contradiction:

Improvefile migration efficiencyVSAvoidpolicy definition flexibility

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent segments the file migration decision into multiple independent probability calculations based on different attributes (age, size, access patterns). Each attribute contributes to the overall migration probability, allowing administrators to define specific criteria for each attribute while maintaining efficient automated migration through the combined probabilistic model.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS7693884B2Managing storage systems based on policy-specific proability

Publication Date: 2010.04.06 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

The invention relates to a method and data processing system of managing a storage system, wherein the storage system comprises a first storage space and a second storage space, wherein the first storage space comprises a plurality of files, wherein for each file of said plurality of files at least one attribute is specified, and wherein a policy is assigned to each attribute. The method comprises the steps of determining a policy-specific probability distribution for each policy and of determining a policy-specific probability for each file by use of the policy-specific probability distribution. The method further comprises the step of determining a total probability for each file by use of each policy-specific probability and of moving each file of the plurality of files having a total probability equal to 1 from the first storage space to the second storage space. The method further comprises the step of moving files of the plurality of files having a total probability between 0 to 1 if the storage space used on the first storage space reaches a high threshold value until a low threshold value is reached, wherein the low threshold value and the high threshold value are predetermined.