IO Path Determination Using Consistent Hashing for Storage Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems, different hosts may select the same controller to process IO requests, leading to unbalanced resource usage and varying IO response speeds, as some controllers become busy while others remain idle.
Innovation Solution
An IO path determination method that uses path grades to direct IO requests to optimal controllers, employing consistent hashing algorithms to select target controllers and threads, and dividing large requests into sub-requests for efficient load balancing and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple hosts send IO requests to the same controller, then the storage system can process requests, but resource usage becomes unbalanced and IO response speed varies
Solution Approach 1:
The patent introduces path grades as a parameter to characterize different IO paths and their grades (host path grade, backend path grade, fault grade). By changing the parameter of path selection from simple round-robin or random to grade-based selection with consistent hashing, the system achieves both high productivity and consistent IO response speed. The path grade tables store grade information that guides request routing decisions.
Solution Approach 2:
The system implements feedback mechanisms by monitoring controller status and dynamically adjusting path selections. When controllers are added or removed, the path grade tables are updated accordingly. The consistent hashing algorithm provides feedback-based load distribution that adapts to system changes while maintaining balanced resource utilization and consistent response times.
2Productivity
If IO requests are distributed to multiple controllers, then resource utilization balances, but system complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-establishing path grade tables that contain all necessary routing information before IO requests arrive. The consistent hashing algorithm pre-determines the mapping between hosts and controllers based on their identifiers. This preliminary setup simplifies the actual request routing process, as the system only needs to lookup pre-computed path grades rather than making complex real-time decisions, thus achieving balanced resource utilization without excessive runtime complexity.
3Reliability
If large data volume requests are processed by a single controller, then request integrity is maintained, but the controller becomes a bottleneck
Solution Approach 1:
The patent applies segmentation by dividing large data volume IO requests into multiple smaller sub-requests. These sub-requests are then distributed to different controllers using the consistent hashing algorithm and path grade tables. Each controller processes a portion of the overall data transfer independently. This segmentation maintains request integrity through proper coordination while eliminating the bottleneck effect, as the total processing throughput becomes the sum of multiple parallel controller capabilities rather than being limited by a single controller.
Data Source
AI summary
An IO path determination method and apparatus, and a device and a readable storage medium. The method comprises: determining a target volume of an IO request, and acquiring a path level of the target volume; when the path level is a host path level, sending the IO request to a preset preferable controller; when the path level is a back-end path level, selecting a target controller from controllers by using a consistent hashing algorithm, and sending the IO request to the target controller; and when the path level is a fault level, determining that a fault occurs in an IO path. An IO path is determined by means of setting different path levels for a volume and in conjunction with a consistent hashing algorithm, such that loads of controllers in a storage system can be effectively balanced.