I/O Splitter for Parallel Storage Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing host processor systems fail to fully utilize parallel I/O capabilities when performing large data operations, as underlying drivers often buffer data into single operations, negating the benefits of parallel processing and requiring application modifications that are not always feasible.
Innovation Solution
The system selects original I/O operations based on specific criteria, subdivides them into multiple operations for parallel execution, and uses a planning tool to predict performance improvements, ensuring that subdivided operations are executed efficiently without further subdivision and reassembled for the driver routines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If drivers buffer data into single I/O operations, then data transfer efficiency is improved, but parallel processing capabilities are not utilized
Solution Approach 1:
The system segments a single large I/O operation into multiple smaller I/O operations that can be executed in parallel. The I/O splitter divides the original I/O request into multiple child I/O operations, each targeting different data portions, enabling parallel processing while maintaining the appearance of a single operation to the application.
Solution Approach 2:
The I/O splitter acts as an intermediary layer between the I/O driver and the storage device. It intercepts I/O operations from the driver, performs segmentation and parallelization, then manages the aggregation of results. This intermediary enables parallel processing without requiring application modifications.
2Device complexity
If applications are modified to perform parallel I/O operations, then parallel processing capabilities are utilized, but application compatibility is reduced
Solution Approach 1:
The I/O splitter serves as an intermediary that transparently enables parallel processing without requiring application modifications. It intercepts I/O operations at the driver level, performs the parallelization logic, and manages result aggregation, making parallel processing available to legacy applications.
Solution Approach 2:
The system performs self-service by automatically detecting when parallelization is beneficial and executing the segmentation logic without application intervention. The I/O splitter monitors I/O operations and autonomously determines whether to split them based on system state and performance criteria.
3Speed
If multiple I/O operations are executed in parallel, then processing speed is improved, but coordination and result aggregation complexity increases
Solution Approach 1:
The system implements feedback mechanisms where the I/O splitter monitors the execution status of child I/O operations and dynamically manages their coordination. It tracks completion status, handles errors, and aggregates results in the correct order, providing feedback control over the parallel execution process.
Solution Approach 2:
The I/O splitter acts as a coordinating intermediary that manages the complexity of parallel operation execution. It handles the bookkeeping of multiple concurrent operations, ensures proper synchronization, and aggregates results before presenting them to the driver, thereby managing coordination complexity centrally.
Data Source
AI summary
Providing I/O operations to a storage device includes selecting a portion of original I/O operations based on a first set of criteria, determining whether to subdivide each of the portion of original I/O operations that are selected according to a second set of criteria different than the first set of criteria, and converting each of the original I/O operations selected for subdivision into a plurality of subdivided I/O operations for different portions of data for a corresponding one of the original I/O operations, where at least two of the different portions are from a single track of data on the storage device. The first set of criteria may include whether the I/O operations are for multiple tracks of data and whether the storage device supports subdividing a single track. The second set of criteria may include determining a measured amount of performance improvement for previous subdivision operations.