I/O Scheduler With Application-Level Awareness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data processing systems lack application-level information awareness, leading to inefficient scheduling of I/O requests, which results in slowed down applications and degraded system throughput when multiple applications access storage concurrently.
Innovation Solution
A data processing system with an interface module that dispatches non-prioritized requests via one data path and transfers application-level information via another, allowing a scheduler to enable access to storage based on this information, enabling prioritization and batching of requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the I/O scheduler uses its own priority scheme to determine the service order of I/O requests, then the scheduling is simple and straightforward, but the system cannot be aware of application-level information, leading to degraded system throughput when multiple applications access storage concurrently
Solution Approach 1:
The patent segments the scheduling information into two distinct paths: a first data path for transmitting I/O requests with basic priority information, and a second data path for transmitting application-level information. This segmentation allows the scheduler to access both simple scheduling priorities and detailed application context without mixing the two information streams, resolving the contradiction between simple scheduling mechanisms and informed decision-making.
Solution Approach 2:
The patent introduces an intermediary layer that collects and processes application-level information from multiple applications and makes it available to the I/O scheduler. This intermediary structure enables the scheduler to access application context without directly embedding complex information gathering logic within the scheduling mechanism itself, thus maintaining scheduling simplicity while enabling informed scheduling decisions.
2Adaptability or versatility
If multiple applications access storage concurrently with interleaved I/O requests, then all applications can utilize storage resources, but the requests are served in arrival order without prioritization, slowing down all tasks and degrading overall system throughput
Solution Approach 1:
The patent implements a feedback mechanism where application-level information (such as application priorities, I/O patterns, and performance requirements) is continuously fed back to the I/O scheduler. This feedback enables the scheduler to dynamically adjust the service order of I/O requests based on current system conditions and application needs, allowing concurrent access while maintaining optimal throughput by prioritizing critical applications.
Solution Approach 2:
The patent makes the scheduling policy dynamic by allowing the I/O scheduler to adjust service orders based on real-time application-level information. Instead of a static first-come-first-served approach, the system dynamically re priorities I/O requests based on changing application requirements, system load, and performance goals, thereby maintaining both concurrent access capability and high system throughput.
Data Source
AI summary
A data processing system comprises a storage device, an interface module and a scheduler. The interface module is configured to dispatch a non-prioritized request via a first data path, and to transfer application-level information of an application via a second data path. The scheduler, coupled to the first and second data path, is configured to enable an access to the storage device according to the non-prioritized request and the application-level information respectively received from the first and second data paths.