I/O Subdivision for Parallel Storage Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current host processor systems fail to fully utilize parallel I/O capabilities for large data operations, as underlying drivers often buffer data into single operations, negating the benefits of parallel processing, and modifying applications to take advantage of this is not practical or possible in many cases.
Innovation Solution
The system selects original I/O operations based on specific criteria, subdivides them into multiple operations for parallel execution, and uses a planning tool to predict performance improvements, allowing for efficient execution without modifying existing applications by leveraging available parallel I/O routes and monitoring performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single I/O operation is used for large amounts of data, then the application logic is simple, but the parallel processing capability of the storage device is not utilized
Solution Approach 1:
The patent divides a single large I/O operation into multiple smaller I/O operations that can be executed in parallel. The I/O operation is segmented based on criteria such as data size thresholds, storage device capabilities, and available parallel channels, allowing the system to utilize multiple processing cores and I/O routes simultaneously while maintaining manageable complexity through automated segmentation logic.
2Productivity
If multiple parallel I/O operations are implemented, then processing efficiency improves, but the number of I/O operations increases
Solution Approach 1:
The patent applies partial action by selectively subdividing only those I/O operations that meet specific criteria (e.g., large data size, available parallel channels, performance thresholds) rather than subdividing all operations. This approach achieves parallel processing efficiency for suitable operations while avoiding the overhead of subdividing operations that would not benefit from parallel execution, thus balancing the quantity of I/O operations with processing efficiency.
3Productivity
If I/O operations are subdivided, then parallel execution is enabled, but the system complexity increases
Solution Approach 1:
The patent implements self-service through automated I/O operation management where the system automatically determines whether to subdivide operations, selects appropriate subdivision strategies, monitors execution progress, and reassembles results without requiring external intervention or complex manual configuration. This automation reduces system complexity by embedding the decision-making logic within the I/O management layer rather than requiring external control mechanisms.
4Productivity
If drivers buffer data into single operations, then data transfer is simplified, but parallel processing benefits are lost
Solution Approach 1:
The patent introduces an intermediary layer between the application/driver and the storage device that intercepts buffered I/O operations, analyzes them against subdivision criteria, and transforms them into parallel operations when appropriate. This intermediary maintains data transfer simplicity for the application layer while enabling parallel processing at the system level, effectively decoupling the simplicity of high-level operations from the complexity of low-level parallel execution.
Data Source
AI summary
Providing I/O operations to a storage device includes selecting a portion of original I/O operations based on a first set of criteria, determining whether to subdivide each of the portion of original I/O operations that are selected according to a second set of criteria different than the first set of criteria, and converting each of the original I/O operations selected for subdivision into a plurality of subdivided I/O operations for different portions of data for a corresponding one of the original I/O operations. The first set of criteria may include whether the I/O operations are for multiple tracks of data. The second set of criteria may include determining a measured amount of performance improvement for previous subdivision operations. Performance of subdivided I/O operations may be monitored. A number of subdivided I/O operations may be limited by a number of available parallel I/O routes.