Storage Defragmentation via Consistency Point Block Relocation

Solution Overview

Problem

Network-based storage systems face performance degradation due to data fragmentation, which is exacerbated by the need for defragmentation processes that compete with client requests and are resource-intensive, especially in heavily loaded systems.

Innovation Solution

A method that involves reading blocks from non-volatile storage, determining fragmentation levels, and relocating them to contiguous locations at a consistency point, using both physical and logical pointers to maintain data integrity and performance without overwriting existing data, allowing for efficient defragmentation without disrupting client operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If defragmentation is performed to reduce data fragmentation, then storage performance is improved, but system resource consumption increases and client request servicing is delayed

Engineering Contradiction:

Improvestorage performanceVSAvoidsystem resource consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system performs defragmentation in advance during consistency points before performance degradation becomes severe. By proactively relocating fragmented blocks during scheduled consistency points rather than reactively during client requests, the system prepares the storage structure ahead of time, reducing the need for intensive defragmentation operations during peak load periods.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements periodic defragmentation at consistency points rather than continuous defragmentation. This periodic approach allows the storage system to maintain fragmentation levels within acceptable thresholds by performing defragmentation cycles at scheduled intervals, balancing performance maintenance with resource consumption.

Inventive Principle:

Principle #19Periodic action

2Productivity

If defragmentation operations are executed during normal operation, then data fragmentation is reduced, but client request servicing is delayed

Engineering Contradiction:

Improvestorage performanceVSAvoidclient request servicing time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs defragmentation in advance during consistency points before performance degradation becomes severe. By proactively relocating fragmented blocks during scheduled consistency points rather than reactively during client requests, the system prepares the storage structure ahead of time, reducing the need for intensive defragmentation operations during peak load periods.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses consistency points as intermediary time windows to perform defragmentation operations. These consistency points act as mediators between normal client request processing and defragmentation activities, allowing block relocation to occur during these intermediate periods without directly impacting ongoing client operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If blocks are relocated to contiguous locations, then read and write performance improves, but data integrity may be compromised if not handled carefully

Engineering Contradiction:

Improveread and write performanceVSAvoiddata integrity

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system uses consistency points as intermediary time windows to perform defragmentation operations. These consistency points act as mediators between normal client request processing and defragmentation activities, allowing block relocation to occur during these intermediate periods without directly impacting ongoing client operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates a mapping between old block locations and new contiguous locations during defragmentation. By maintaining this mapping and using it to redirect access patterns, the system ensures that data integrity is preserved while achieving contiguous storage. The mapping acts as a copy of the data's logical structure, allowing physical relocation without loss of logical relationships.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS7562203B2Storage defragmentation based on modified physical address and unmodified logical address

Publication Date: 2009.07.14 NETAPP INC

AI summary

A method in a storage system reads blocks having both physical addresses and logical addresses in the storage system from a first set of locations in non-volatile storage to a cache memory and determines the level of data fragmentation in the non-volatile storage. If the level of data fragmentation exceeds a threshold level, the cache memory is marked and the blocks are written to physically sequential locations with new physical addresses in the non-volatile storage at a data consistency point without changing the logical addresses of the blocks.