Distributed Storage Traffic Scheduling for Write Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing traffic scheduling methods for distributed storage clusters are inefficient in balancing write capacities and loads across storage pools, relying on manual experience that leads to poor accuracy and prolonged balancing times.
Innovation Solution
Implementing an AI-driven traffic scheduling method that predicts future write traffic and load balances using capacity and load prediction quantization models, generating a target traffic scheduling policy to quickly align capacities and loads across storage pools.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If manual experience-based adjustment of traffic write proportions is used, then the scheduling method is simple to implement, but the accuracy of adjusting traffic write proportions is poor and scheduling efficiency is low
Solution Approach 1:
The patent replaces the manual mechanical adjustment process with an automated intelligent system. A computing device automatically collects write capacity and load data from multiple storage pools, predicts future traffic patterns using machine learning models, and generates optimized traffic scheduling policies without human intervention, thereby improving accuracy while maintaining implementation simplicity
Solution Approach 2:
The system enables self-service scheduling where the storage cluster automatically monitors its own performance metrics, predicts future states, and adjusts traffic distribution autonomously. The intelligent algorithm continuously learns from historical data and makes real-time scheduling decisions, allowing the system to self-optimize without external manual control
2Ease of operation
If manual experience-based adjustment of traffic write proportions is used, then the implementation process is straightforward, but it takes multiple periods to balance write capacities and loads
Solution Approach 1:
The patent applies preliminary action by predicting future write traffic patterns and capacity needs before they actually occur. The system uses historical data to forecast upcoming traffic surges or capacity constraints, allowing the scheduling algorithm to proactively adjust traffic distribution in advance, thereby achieving balance faster and reducing the time lost during balancing operations
Solution Approach 2:
The system implements dynamic scheduling where traffic write proportions are continuously adjusted based on real-time conditions and predictions. Rather than static manual settings, the intelligent algorithm dynamically optimizes traffic distribution across storage pools, adapting to changing workloads and capacity availability to achieve rapid balancing
3Productivity
If AI-driven prediction models are used, then scheduling efficiency is significantly improved, but the system complexity increases
Solution Approach 1:
The patent introduces an intermediary intelligent scheduling layer between the storage pools and traffic sources. This intermediate system handles the complexity of data collection, prediction, and optimization, presenting a simple interface for traffic management while managing sophisticated algorithms internally, thus improving efficiency without proportionally increasing observable system complexity
Data Source
AI summary
Embodiments of the present disclosure provide a traffic scheduling method for a distributed storage cluster including: acquiring a write traffic corresponding to each of N historical periods; acquiring a write traffic proportion, a capacity increment and a load corresponding to each of the storage pools in each of the historical periods; predicting a write traffic prediction value of an (N+1)th new period based on the write traffic; generating a capacity increment prediction quantization model and a load prediction quantization model based on the write traffic, the write traffic proportion, the capacity increment and the load; and invoking the capacity increment prediction quantization model and the load prediction quantization model based on the write traffic prediction value to generate a target traffic scheduling policy of the (N+1)th new period, and controlling traffic scheduling of each resource pool based on a target scheduling proportion indicated by the target traffic scheduling policy.