Striping Table Optimization for Network Storage Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network storage systems face inefficiencies due to uneven data distribution and the need for frequent data migration when the number of volumes changes, leading to overburdened nodes and performance degradation.
Innovation Solution
An iterative process for optimizing a striping table that balances multiple performance metrics, such as evenness of storage consumption, hot spots, and adaptability to changes in the storage system geometry, by successively adding regions and replacing cells with new volume identifiers, minimizing the number of changes required when volumes are added or removed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is distributed across multiple storage server nodes, then processing load is reduced and system scalability is improved, but individual nodes may become overburdened when servicing heavily accessed data containers
Solution Approach 1:
The patent segments the storage system into multiple storage server nodes, each responsible for specific data containers. By dividing the monolithic storage system into smaller independent units, the system can distribute processing load across nodes while maintaining the ability to handle heavy access patterns through replication and load balancing mechanisms.
Solution Approach 2:
The patent implements local quality by allowing different storage server nodes to have different roles and characteristics based on their workload. Nodes servicing heavily accessed data containers can be configured with higher performance characteristics or additional replication factors, while less busy nodes can handle standard workloads, optimizing overall system performance and reliability.
2Productivity
If a striped file system is used to distribute data across multiple volumes, then load balancing is improved, but the striping table becomes tied to a specific number of volumes requiring data migration when geometry changes
Solution Approach 1:
The patent implements dynamic striping tables that can automatically adapt to changes in storage geometry. When volumes are added or removed from the storage system, the striping table dynamically recalculates the distribution pattern without requiring complete data migration. This allows the system to maintain optimal load balancing while being flexible to storage capacity changes.
Solution Approach 2:
The patent changes the parameter of the striping table from being fixed to a specific number of volumes to being dynamic and adaptable. The striping algorithm modifies its behavior based on the current number of available volumes, using modular arithmetic and redistribution strategies to maintain even data distribution without requiring data migration during geometry changes.
3Manufacturing precision
If the striping table is modified when volumes are added or removed, then the data distribution remains valid, but significant data migration is required consuming computing resources and bandwidth
Solution Approach 1:
The patent performs preliminary actions by pre-calculating and pre-positioning data during initial striping operations. When storage geometry changes occur, the system uses the existing striping pattern as a foundation and only migrates the minimal necessary data to new volumes, rather than performing a complete redistribution. This preliminary organization of data significantly reduces migration requirements during geometry changes.
Solution Approach 2:
The patent applies partial action by migrating only the necessary portion of data when storage geometry changes. Instead of migrating all data during volume additions or removals, the system identifies and migrates only the specific data blocks that need to be redistributed, leaving the majority of data in place. This partial migration approach maintains data distribution accuracy while minimizing time and resource consumption.
Data Source
AI summary
The technique introduced here includes generating a data structure for use in determining how responsibilities for services should be distributed amongst a plurality of resources in a network storage system. The technique includes an iterative process of optimizing the data structure for a plurality of performance/quality metrics, such as evenness of storage consumption across the storage system or a designated subset thereof, number of “hot spots”, degree of data scatter, and number of changes needed to reflect a change in storage system geometry. The data structure can be a striping table for striping logical containers of data across multiple storage resources, such as data volumes, or physical storage devices. The “responsibilities for services”, can include responsibility for storing a stripe of a logical container of data or responsibility for storing a segment of parity data for a logical container of data.