Dynamic Thread Allocation for Tail Latency SLO in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face challenges in meeting tail latency requirements for latency-sensitive applications due to resource competition and inflexibility in thread models, leading to low resource utilization and interference from batch-job applications.
Innovation Solution
A window-based dynamic thread resources allocation method is introduced, where request queues for latency-sensitive applications are divided into windows, and threads are dynamically adjusted based on real-time monitoring of request processing and arrival rates to ensure target tail latency requirements, with backup threads handling IO operations to prevent delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If latency-sensitive applications are deployed separately to monopolize system resources, then tail latency requirements are met, but resource utilization rate decreases
Solution Approach 1:
The patent implements dynamic thread resource allocation where the number of threads allocated to latency-sensitive applications changes over time based on system conditions. The thread pool size is adjusted dynamically using formulas that consider SLO compliance status, request queue lengths, and system load, allowing the system to transition between guaranteeing tail latency and maximizing resource utilization
2Productivity
If latency-sensitive applications are collocated with batch-job applications, then resource utilization rate improves, but tail latency is affected due to resource competition
Solution Approach 1:
The patent segments thread resources into different pools: dedicated threads for latency-sensitive applications and shared threads for batch-job applications. This segmentation allows both application types to coexist on the same physical system while ensuring that latency-sensitive workloads have guaranteed thread availability, preventing resource competition from affecting their tail latency
Solution Approach 2:
The patent applies different resource allocation strategies to different application types. Latency-sensitive applications receive prioritized thread allocation with SLO-based guarantees, while batch-job applications utilize remaining shared resources. This local differentiation in resource quality ensures tail latency requirements are met for critical applications while allowing non-critical applications to share resources
3Reliability
If fixed number of threads are allocated to latency-sensitive applications, then tail latency SLO is guaranteed, but system resource utilization rate decreases
Solution Approach 1:
The patent replaces fixed thread allocation with dynamic adjustment mechanisms. The number of threads allocated to latency-sensitive applications is continuously adjusted based on SLO compliance monitoring, request arrival rates, and system conditions. When SLO is comfortably met, threads are released to increase utilization; when SLO is at risk, threads are allocated to ensure compliance
4Productivity
If threads are dynamically adjusted based on real-time monitoring, then resource utilization rate improves, but system complexity increases
Solution Approach 1:
The patent implements feedback-based thread allocation where the system continuously monitors SLO compliance status, request queue lengths, and thread utilization metrics. This feedback drives automatic thread pool adjustment, with the system responding to performance measurements by allocating or releasing threads. The feedback mechanism enables utilization improvement through automated, data-driven decisions rather than static configuration
Data Source
AI summary
The present invention proposes a dynamic resources allocation method and system for guaranteeing tail latency SLO of latency-sensitive applications. A plurality of request queues is created in a storage server node of a distributed storage system with different types of requests located in different queues, and thread groups are allocated to the request queues according to logical thread resources of the service node and target tail latency requirements, and thread resources are dynamically allocated in real time, and the thread group of each request queue is bound to physical CPU resources of the storage server node. The client sends an application's requests to the storage server node; the storage server node stores the request in a request queue corresponding to its type, uses the thread group allocated for the current queue to process the application's requests, and sends responses to the client.