Segmented Global File System with ILM for Scalable Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Information Lifecycle Management (ILM) approaches are limited to direct-attached or SAN-attached storage systems and traditional, non-segmented distributed file systems, which restrict their scalability and configuration ease in larger networks.
Innovation Solution
A global file system, specifically a segmented file system, is integrated with ILM to create a distributed data storage system that allows for easy configuration, high scalability, and automated data movement among servers, using a network-attached storage system with nodes organized by cost-performance characteristics and dynamic management of storage resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized metadata management approach is used in a global file system, then namespace management is simplified, but system performance deteriorates due to centralized bottlenecks
Solution Approach 1:
The patent divides the centralized metadata management into distributed metadata servers, where each metadata server manages a portion of the namespace. This segmentation allows the system to maintain simplified namespace management through a unified view while improving performance by distributing the metadata management load across multiple servers, eliminating the centralized bottleneck.
2Quantity of substance
If storage resources are added to a distributed file system, then storage capacity increases, but namespace management complexity increases
Solution Approach 1:
The namespace is divided into segments that can be independently managed by different metadata servers. When storage resources are added, new namespace segments can be created and assigned to appropriate metadata servers without affecting existing segments, allowing storage capacity to increase while maintaining manageable namespace complexity through modular organization.
Solution Approach 2:
The patent implements a universal namespace management mechanism that can handle multiple types of storage resources (files, directories, symbolic links) and multiple namespace operations (creation, deletion, modification) through a unified interface. This universality allows storage capacity to expand while maintaining consistent management procedures, preventing complexity from increasing proportionally with capacity.
3Productivity
If ILM moves data between different storage devices, then storage optimization is achieved, but data movement overhead increases
Solution Approach 1:
The system performs preliminary classification of data according to ILM policies, assigning data to appropriate storage devices based on access patterns and requirements. By pre-classifying data and establishing movement policies in advance, the system optimizes storage allocation without requiring frequent real-time data movement, thereby reducing data movement overhead while maintaining storage optimization.
Solution Approach 2:
The ILM system automatically monitors data access patterns and triggers data movement between storage devices based on predefined policies without requiring manual intervention. This self-service mechanism optimizes storage utilization by automatically moving data to appropriate devices while minimizing overhead through automated decision-making and execution, reducing the time and resources required for manual data management.
Data Source
AI summary
A networked storage system includes network-attached file system (FS) nodes implementing a distributed global file system. Each FS node includes a file system server and at least one storage system including storage resources (such as disk drives) representing portions of a global file system storage space. The FS nodes are organized according to respective cost-performance characteristics of the storage systems, generally ranging from a high-cost, high-performance characteristic to a low-cost, low-performance characteristic. A storage management system performs information lifecycle management (ILM) which includes allocating the FS nodes for storing data according to a mapping between an ILM-based data classification scheme and the cost-performance characteristics of the storage devices, as well as dynamically managing the placement of data among the storage devices according to the ILM-based classification of the data. The storage management system also performs global file system management including allocating a set of identifiers (e.g. segment identifiers) of the distributed global file system among the FS nodes, and dynamically adjusting the allocation of the identifiers among the FS nodes in response to the addition and removal of whole storage systems and/or storage resources within each of the storage systems.