Block Storage Organization via Filesystem Metadata

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional storage systems that separate file-based and block-based functionalities over a network often store data suboptimally, leading to fragmentation, which limits read performance due to random data block placement.

Innovation Solution

The solution involves communicating metadata from a file server to a block storage server to organize data blocks belonging to the same files near each other in block storage, improving read performance by logically rearranging and compressing data blocks based on filesystem-level metadata.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If data blocks are written to block storage in random order, then write flexibility is improved, but read performance deteriorates due to fragmentation

Engineering Contradiction:

Improvewrite flexibilityVSAvoidread performance

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The file server performs preliminary actions by determining the optimal placement of data blocks in the filesystem before transmitting them to the block storage server. The metadata containing this placement information is sent along with the data blocks, enabling the block storage server to pre-organize blocks in the correct sequential order, thus ensuring both write flexibility and optimal read performance

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If file-based and block-based functionalities are separated over a network, then system modularity is improved, but data organization quality deteriorates

Engineering Contradiction:

Improvesystem modularityVSAvoiddata organization quality

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The file server provides feedback to the block storage server by transmitting metadata that describes the optimal placement of data blocks in the filesystem. This feedback mechanism enables the block storage server to organize data blocks according to filesystem requirements, maintaining high data organization quality despite the separation of file-based and block-based functionalities over the network

Inventive Principle:

Principle #23Feedback

3Device complexity

If data blocks are stored without organization information, then storage simplicity is improved, but storage efficiency deteriorates due to fragmentation

Engineering Contradiction:

Improvestorage simplicityVSAvoidstorage efficiency

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

Metadata acts as an intermediary between the file server and block storage server. It carries placement information that enables the block storage server to organize data blocks efficiently without requiring complex changes to the storage system itself. This intermediary mechanism improves storage efficiency while maintaining relative simplicity in the storage architecture

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11513739B2File layer to block layer communication for block organization in storage

Publication Date: 2022.11.29 EMC IP HLDG CO LLC

AI summary

A method performed by a block-storage server, of storing data is described. The method includes (1) receiving, from a remote file server, data blocks to be written to persistent block storage managed by the block-storage server; (2) receiving, from the remote file server, metadata describing a placement of the data blocks in a filesystem managed by the remote file server; and (3) organizing the data blocks within the persistent block storage based, at least in part, on the received metadata. An apparatus, system, and computer program product for performing a similar method are also provided.