Storage Server Workload Management via Disk Utilization Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in managing I/O requests due to First-In-First-Out processing, leading to lower priority requests being processed before higher priority ones, especially when multiple databases share storage resources, resulting in performance and response time issues and requiring manual over-provisioning or scheduling adjustments.
Innovation Solution
Implementing a storage server that tracks disk usage levels to compute I/O costs and enforce precise workload limits by scheduling I/O requests based on logical characteristics such as priority, user identity, and consumer groups, using scheduling and selection policies to optimize resource allocation and processing order.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If storage systems process I/O requests on a First-In-First-Out basis, then the processing order is simple and straightforward, but lower priority requests are processed before higher priority ones, leading to performance degradation
Solution Approach 1:
The patent implements dynamic I/O request scheduling where the processing order is not fixed but adapts based on request priorities. The storage system dynamically reorders I/O requests in the queue based on their priority levels, allowing high-priority requests to be processed ahead of low-priority ones, thus resolving the contradiction between simple FIFO processing and priority-based performance optimization
Solution Approach 2:
The patent changes the processing parameter from simple temporal order (FIFO) to priority-based ordering. By introducing priority as a sorting criterion, the system transforms the rigid First-In-First-Out mechanism into a flexible priority-scheduled mechanism that can adaptively reorder requests based on their importance, thereby improving processing efficiency without complex manual intervention
2Adaptability or versatility
If multiple databases share storage resources, then resource utilization is improved and costs are reduced, but performance and response time problems occur between applications
Solution Approach 1:
The patent segments the shared storage resources by introducing virtual storage pools and logical partitions. Each database or application can be allocated specific virtual storage resources while physically sharing the same underlying storage infrastructure. This segmentation allows multiple databases to coexist without performance interference, as each operates within its designated virtual boundary
Solution Approach 2:
The patent introduces a storage virtualization layer as an intermediary between multiple databases and the physical storage system. This intermediary layer manages I/O requests from multiple applications, enforces resource allocation policies, and prevents any single application from monopolizing storage resources, thereby maintaining both sharing benefits and performance isolation
3Productivity
If storage systems reorder I/O requests to improve disk efficiency, then disk utilization is optimized, but the reordering is performed without regard for the purpose behind the requests, processing lower priority I/O before higher priority I/O
Solution Approach 1:
The patent modifies the sorting parameters of I/O request reordering by incorporating priority levels as a primary sorting criterion. Instead of reordering based solely on disk efficiency metrics, the system now considers both efficiency and priority, ensuring that high-priority requests are not displaced below low-priority ones during the reordering process, thus maintaining reliability while optimizing disk utilization
4Productivity
If manual scheduling adjustments are made to mitigate performance problems, then critical query performance is improved, but the process becomes laborious and impossible when databases have different administrators
Solution Approach 1:
The patent implements automated workload management and resource allocation mechanisms that operate without manual intervention. The storage system automatically monitors performance metrics, identifies priority conflicts, and adjusts scheduling policies dynamically. This self-service approach eliminates the need for manual scheduling adjustments and works seamlessly across databases with different administrators, as the automation handles coordination and conflict resolution
Data Source
AI summary
Disclosed is an improved approach for managing access to resources by workloads in a computing system. A much more accurate and useful technique is provided for determining disk utilization, and for using the calculated disk utilization to enforce workload constraints and limits. The technique may be used by any application that is attempting to share storage between multiple workloads can use the present solution, as well as any operating system and workload manager that need to manage workloads and resources.