Adaptive Thread Pool for Storage Quota Checking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional resource quota checking in storage systems is inefficient due to excessive thread usage during high workloads, leading to delayed processing of other IO requests, and underutilization of resources during low workloads, resulting in inconsistent quota updates and prolonged processing times.
Innovation Solution
Adaptive allocation of threads for resource quota checking tasks based on current workload, adjusting the number of processes between predetermined thresholds to optimize resource utilization and response speed, with dynamic adjustments every 10 minutes and prioritization of tasks based on workload and execution time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed number of threads (8 threads) is used to perform quota checking concurrently, then the scanning speed is accelerated, but during high workload periods, this causes delayed processing of other IO requests
Solution Approach 1:
The patent implements dynamic thread pool adjustment where the number of threads is not fixed but adapts based on system workload conditions. The thread pool size is adjusted within a range (minimum to maximum threads) based on real-time monitoring of IO request queues and system load, allowing the system to accelerate quota checking when resources are available while preventing delays to other IO operations during high workload periods
Solution Approach 2:
The patent changes the parameter of thread pool size dynamically based on system conditions. By monitoring workload metrics and adjusting the number of concurrent threads accordingly, the system optimizes the balance between quota checking speed and overall IO processing performance, avoiding the fixed parameter limitation of traditional approaches
2Reliability
If the thread pool size is reduced to prevent IO request delays, then system stability is maintained, but resource utilization becomes underutilized during low workload periods
Solution Approach 1:
The system dynamically adjusts thread pool size based on real-time workload monitoring, expanding the thread pool when system load is low to maximize resource utilization and contracting it when load is high to maintain stability. This dynamic adaptation allows the system to achieve both reliability and productivity across different operating conditions
Solution Approach 2:
The patent implements a feedback mechanism that continuously monitors system workload, IO request queue depth, and thread execution status. Based on this feedback, the system automatically adjusts the thread pool size to optimize both stability and resource utilization, preventing both over-utilization and under-utilization scenarios
3Measurement precision
If the quota checking task scans the whole file system to acquire initial quota, then accurate quota information is obtained, but the task takes a long time to finish
Solution Approach 1:
The patent segments the file system scanning task into multiple concurrent thread operations. Instead of a single thread scanning the entire file system sequentially, multiple threads divide and conquer different portions of the file system simultaneously, maintaining accurate quota collection while significantly reducing the total scanning time through parallel processing
Solution Approach 2:
The system employs partial scanning strategies where threads may scan portions of the file system or use sampling techniques to estimate quota usage, providing sufficiently accurate results without completing exhaustive scans of every inode, thereby reducing overall processing time while maintaining acceptable precision
Data Source
AI summary
Techniques perform resource configuration. The techniques involve determining, for a resource quota checking task for a file system on a storage system, a current workload of the storage system; in response to the workload exceeding a predetermined upper threshold, performing the resource quota checking task with a first number of processes; and in response to the workload being below a predetermined lower threshold, performing the resource quota checking task with a second number of processes, the second number being greater than the first number.