Global Namespace Distribution for Cluster Filesystem Scaling

Solution Overview

Problem

In cluster file systems, achieving linear performance scaling is challenging due to uneven file distribution, which leads to synchronization and crash resilience issues.

Innovation Solution

The distribution of files across cluster nodes is managed through a global namespace abstraction, where namespace trees are linked through a centralized access point, enabling a unified virtualized global filesystem and load balancing across nodes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If files are distributed across cluster nodes without global namespace management, then storage capacity scales linearly, but file distribution becomes uneven leading to synchronization and crash resilience issues

Engineering Contradiction:
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The global namespace is segmented into distributed namespace trees across multiple cluster nodes, with each node maintaining a portion of the namespace hierarchy. This segmentation allows linear performance scaling while the centralized access point ensures coordinated access and maintains crash resilience through structured distribution.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

A centralized access point is introduced as an intermediary between clients and the distributed namespace trees. This intermediary coordinates namespace operations, ensures consistent file distribution across nodes, and maintains crash resilience by managing access patterns and preventing uneven distribution issues.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If a centralized namespace is used across all cluster nodes, then crash resilience is improved, but performance scaling is limited due to synchronization overhead

Engineering Contradiction:
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The centralized namespace concept is segmented into distributed namespace trees, where each node maintains local namespace structures. This reduces synchronization overhead by allowing independent operation at the node level while maintaining overall namespace consistency through the centralized access point, enabling both crash resilience and linear performance scaling.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The namespace structure transitions from a single centralized dimension to a multi-dimensional distributed structure with namespace trees across multiple nodes. This dimensional change allows the system to achieve both centralized coordination benefits for reliability and distributed independence for performance scaling.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

3Device complexity

If files are distributed unevenly across cluster nodes, then device complexity is reduced, but synchronization issues and performance degradation occur

Engineering Contradiction:
Core Design Contradiction:
Device complexityVSProductivity

Solution Approach 1:

The centralized access point acts as an intermediary that manages file distribution across namespace trees, ensuring even distribution without requiring complex node-to-node coordination. This intermediary approach maintains simple device complexity at each node while achieving balanced performance scaling across the cluster.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements feedback mechanisms through the centralized access point that monitors file distribution across namespace trees and adjusts allocation patterns. This feedback loop ensures even file distribution is maintained automatically, preventing performance degradation while keeping individual node complexity low.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS10762050B2Distribution of global namespace to achieve performance and capacity linear scaling in cluster filesystems
Publication Date: 2020.09.01
  • US10762050B2 patent drawing
  • US10762050B2 patent drawing
  • US10762050B2 patent drawing

AI summary

The distribution of global namespace to achieve performance and capacity linear scaling in cluster filesystems. Specifically, the disclosure entails the distribution of files, tracked in namespace trees, among various cluster nodes while linking the namespace trees through a centralized access point, thus providing a global namespace abstraction for the distributed cluster.