Multi-Pathing Optimization Module for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face performance degradation when a single data container is heavily accessed, leading to overburdening of resources, as existing methods for balancing bandwidth and processing load across multiple storage systems primarily utilize the cluster switching fabric, which is less optimal compared to the high bandwidth system bus within each node.
Innovation Solution
A Multi-Pathing Optimization Module (MPOM) is implemented on the client or between the client and storage system, which constructs a Location Data Structure (LDS) containing striping rules and indices of N-blades and D-blades connected via a high bandwidth system bus, allowing data access requests to be routed directly through the most optimal path, utilizing the high bandwidth connection between N and D-blades instead of the cluster switching fabric.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data access requests are routed through the cluster switching fabric to balance load across multiple storage systems, then bandwidth utilization is improved, but the system bus within each node is underutilized and performance is degraded
Solution Approach 1:
The patent pre-establishes location data structures (LDS) that map data containers to specific storage systems and system bus paths before data access occurs. This preliminary mapping enables direct routing through the high-bandwidth system bus without real-time path selection complexity, resolving the contradiction between performance optimization and system complexity.
Solution Approach 2:
The patent introduces a location data structure (LDS) as an intermediary layer between the client and storage systems. This LDS contains pre-computed routing information that mediates the path selection process, enabling direct system bus access without requiring complex real-time routing decisions through the cluster switching fabric.
2Reliability
If multiple storage systems are used to store a single data container, then reliability is improved, but resource overburdening occurs when heavily accessed containers are concentrated on fewer systems
Solution Approach 1:
The patent implements data striping that distributes different portions (stripes) of a single data container across multiple storage systems with different failure domains. This local distribution ensures that if one storage system fails, other stripes remain accessible, improving reliability while preventing resource overburdening through balanced distribution.
Solution Approach 2:
The patent divides data containers into multiple stripes that are independently stored across different storage systems. This segmentation allows the system to maintain high reliability through distribution while improving resource utilization efficiency by balancing the storage burden across multiple systems rather than concentrating it on fewer systems.
3Ease of operation
If the cluster switching fabric is used for data access, then system simplicity is maintained, but bandwidth capacity is underutilized compared to the high bandwidth system bus
Solution Approach 1:
The patent pre-computes and stores optimal routing paths in location data structures before data access occurs. This preliminary action identifies which storage systems have data stripes and which system bus paths should be used, enabling direct high-bandwidth access without requiring complex real-time routing decisions, thus maintaining routing simplicity while maximizing bandwidth utilization.
Data Source
AI summary
A Multi-Pathing Optimization Module (MPOM) interfaces with a client to route a data access request along a most optimal path to a storage system serving data container stored on a striped volume set (SVS). The MPOM constructs and references a Location Data Structure (LDS) containing striping rules associated with the SVS, a list of network elements (N-blades) operatively connected to the client, and a list of disk elements (D-blades) sharing a node of the storage system with the listed N-blades. The MPOM executes a Client_Locate( ) function on a per data access request basis to identify the D-blade serving the volume which will receive the request. When possible, the MPOM routes the data access request directly to the N-blade sharing a node with the identified D-blade, thereby using the most optimal path from the client to the SVS.