Erasure Coding Tiering Using Traffic Prediction and PID Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Modern storage systems face challenges in dynamically managing data with varying access patterns, as static rules struggle to maintain efficient data access and storage capabilities in rapidly changing environments.

Innovation Solution

A computerized method using a traffic estimation model and a PID controller to dynamically determine erasure coding formats for data extents, integrating predictive analytics and localized decision-making to optimize storage capacity and access efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If static rules are used to determine EC format conversion, then implementation simplicity is maintained, but adaptability to rapidly changing data access patterns deteriorates

Engineering Contradiction:

Improveimplementation simplicityVSAvoidadaptability to changing data access patterns

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic EC format selection by replacing static rules with a predictive model that continuously adapts to changing data access patterns. The system uses historical traffic data to forecast future access patterns and dynamically determines optimal EC formats, enabling the storage system to respond flexibly to rapidly changing workloads while maintaining implementation feasibility through automated decision-making.

Inventive Principle:

Principle #15Dynamics

2Speed

If hot EC formats are used for frequently accessed data, then data accessibility is improved, but storage resource consumption increases

Engineering Contradiction:

Improvedata accessibilityVSAvoidstorage resource consumption

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The patent changes the storage system parameters by dynamically selecting between different EC formats (hot vs. cold) based on predicted data accessibility requirements. The system uses a predictive model to forecast future access patterns and adjusts the EC format parameters accordingly, allowing frequently accessed data to use hot formats for fast access while less frequently accessed data uses cold formats to conserve storage resources.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent applies different EC format qualities to different data extents based on their individual access patterns. Instead of using a uniform EC format for all data, the system evaluates each data extent's historical traffic and predicts its future accessibility requirements, then assigns the appropriate EC format (hot or cold) to each extent, optimizing the balance between accessibility and resource consumption on a per-extent basis.

Inventive Principle:

Principle #3Local quality

3Quantity of substance

If cold EC formats are used to reduce storage resources, then storage efficiency is improved, but data access performance deteriorates

Engineering Contradiction:

Improvestorage resource efficiencyVSAvoiddata access performance

Core Design Contradiction:

Quantity of substanceVSSpeed

Solution Approach 1:

The patent performs preliminary action by using a predictive model to forecast future data access patterns before actual access occurs. This allows the system to proactively select appropriate EC formats in advance, ensuring that data likely to be frequently accessed is stored in hot formats with better access performance, while data predicted to be less frequently accessed is stored in cold formats for storage efficiency.

Inventive Principle:

Principle #10Preliminary action

4Device complexity

If static EC format rules are applied, then system complexity is reduced, but responsiveness to rapidly occurring changes deteriorates

Engineering Contradiction:

Improvesystem complexityVSAvoidresponsiveness to system changes

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent implements feedback control by using historical traffic data to train and update a predictive model that continuously learns from actual system behavior. The model receives feedback in the form of historical access patterns and uses this information to improve future predictions, enabling the system to respond dynamically to rapidly occurring changes while maintaining manageable complexity through automated learning and adaptation.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20260072596A1Data tiering with erasure coding using traffic estimation and feedback control

Publication Date: 2026.03.12 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A computerized method determines erasure coding (EC) formats for data extents in storage systems. Traffic history data of a data extent to be stored in the storage system is obtained. Estimated future traffic data is generated using a traffic estimation model, wherein the traffic history data is provided to the model as input. A current storage system constraint is calculated using the obtained traffic history data, and a constraint adjustment value is determined using a proportional, integral, derivative (PID) controller based on the current storage system constraint. An EC format for the data extent is determined using the estimated future traffic data and the determined constraint adjustment value and the data extent is stored in the storage system using the determined EC format. Use of the model and the PID controller balances optimized storage use with efficient data traffic handling.