Metadata Block Redistribution for Storage Utilization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current storage systems face challenges in maintaining optimal metadata storage utilization rates, leading to inefficiencies and errors due to excessive metadata storage in a single transaction, which can cause delays and data loss.

Innovation Solution

A method that determines valid metadata rates in metadata blocks, selects blocks with lower rates, and redistributes valid metadata across multiple blocks, allowing for incremental storage and reducing the need for frequent compression, thereby maintaining a reasonable utilization rate and improving system performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If all metadata is stored into the metadata storage area in one transaction, then the metadata storage is completed, but the metadata storage area utilization rate becomes too high causing errors and data loss

Engineering Contradiction:

Improvemetadata storage completionVSAvoiddata loss prevention

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent divides the metadata storage process into multiple transactions, storing metadata in batches rather than all at once. Each transaction stores a portion of metadata, preventing the metadata storage area from becoming overloaded. This segmentation approach maintains reliability by avoiding the high utilization rate that causes errors and data loss, while still achieving complete metadata storage through iterative processes.

Inventive Principle:

Principle #1Segmentation

2Quantity of substance

If metadata storage area utilization rate is kept high, then storage capacity is maximized, but errors occur due to excessive metadata storage in single transaction

Engineering Contradiction:

Improvemetadata storage capacityVSAvoiderror reduction

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The patent implements segmentation by dividing large-scale metadata storage into multiple smaller transactions. Instead of storing all metadata in one transaction which causes errors, the system stores metadata in batches, keeping each transaction's utilization rate within a safe range. This approach maintains overall storage capacity while preventing errors through controlled incremental storage.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary actions by pre-calculating the utilization rate threshold and determining batch sizes before storage operations. The system proactively monitors the metadata storage area utilization rate and adjusts storage operations accordingly, preventing errors before they occur rather than reacting after problems arise.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If compression is performed frequently to maintain low utilization rate, then data loss is reduced, but system performance deteriorates due to excessive compression operations

Engineering Contradiction:

Improvedata loss preventionVSAvoidsystem performance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements a feedback mechanism that continuously monitors the metadata storage area utilization rate and adjusts compression operations accordingly. The system performs compression only when the utilization rate exceeds a predetermined threshold, avoiding unnecessary compression operations. This feedback-based approach maintains reliability by preventing data loss through controlled compression while preserving system performance by minimizing redundant compression operations.

Inventive Principle:

Principle #23Feedback

4Reliability

If incremental storage is used to maintain reasonable utilization rate, then data loss is reduced, but the number of storage transactions increases

Engineering Contradiction:

Improvedata loss preventionVSAvoidnumber of transactions

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent accepts the trade-off of increased transaction numbers in exchange for reliability. By segmenting metadata storage into incremental batches, the system prevents data loss that would occur with single large transactions. The increased number of transactions is a necessary consequence of maintaining safe utilization rates, and the system optimizes this by using efficient batch processing and threshold-based control.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11287993B2Method, device, and computer program product for storage management

Publication Date: 2022.03.29 EMC IP HLDG CO LLC

AI summary

Techniques involve: determining corresponding valid metadata rates of a plurality of metadata blocks stored in a metadata storage area of a storage system, the valid metadata rate of each metadata block indicating a ratio of valid metadata in the metadata block to all metadata in the metadata block; selecting a predetermined number of metadata blocks having a valid metadata rate lower than a first valid metadata rate threshold from the plurality of metadata blocks; storing valid metadata in the predetermined number of metadata blocks into at least one metadata block following the plurality of metadata blocks in the metadata storage area; and making the valid metadata in the predetermined number of metadata blocks invalid. Accordingly, such techniques can improve the efficiency of the storage system.