Adaptive Data Block Merging for Lower Distributed I/O Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, concurrent data operations are hindered when data blocks are smaller than the smallest segmentation unit, leading to insufficient system performance and high latency, affecting user experience and system stability.
Innovation Solution
A data I/O processing method that collects time delay statistical records during data input/output, sets a rational waiting time period based on these records, and merges data blocks within this period to optimize data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data blocks are processed sequentially without merging, then system simplicity is maintained, but data I/O time increases and throughput decreases
Solution Approach 1:
The patent merges multiple small data blocks into larger consolidated blocks before I/O operations. The data processing method collects time delay statistical records for each data block, determines waiting time periods, and merges data blocks within these periods to optimize I/O efficiency. This merging strategy reduces the number of separate I/O operations and improves throughput while managing I/O time through controlled consolidation.
Solution Approach 2:
The patent performs preliminary actions by collecting time delay statistical records and determining waiting time periods before actual data I/O operations occur. The system pre-processes data blocks by merging them within calculated waiting periods, so that when I/O operations are executed, the data is already optimized for efficient transfer. This preliminary consolidation reduces I/O time and improves productivity.
2Productivity
If data blocks are merged within waiting time periods, then data I/O time is reduced and throughput is enhanced, but system complexity increases
Solution Approach 1:
The patent implements feedback mechanisms by collecting time delay statistical records for each data block and using this information to determine waiting time periods. The system continuously monitors actual I/O completion times, compares them with expected times, and adjusts the merging strategy accordingly. This feedback loop allows the system to optimize data block merging dynamically, improving throughput while managing complexity through adaptive control.
3Productivity
If concurrent operations are enabled for data blocks, then system performance is improved, but data consistency and stability may be compromised
Solution Approach 1:
The patent applies dynamics by making the data block merging strategy adaptive and flexible. The waiting time periods are dynamically adjusted based on actual time delay statistical records, allowing the system to optimize concurrent operations while maintaining stability. The merging process is dynamically controlled, enabling the system to respond to varying workload conditions and maintain reliability through flexible, rather than rigid, concurrent processing.
Data Source
AI summary
A data I/O processing method includes obtaining data requested consisting of a plurality of basic data blocks in sequence; grouping the data in sequence to obtain a plurality of segmented data blocks in sequence; sequentially determining whether each segmented data block has a time delay statistical record based on the time for completion of the operation processing of a previous basic data block; in response to the time delay statistical record being present in the segmented data block, setting a waiting time period according to the time delay statistical record; sequentially merging the basic data blocks, which have not been subjected to the operation processing, in the segmented data blocks within the waiting time period until the waiting time period is ended or the merged basic data blocks reach the size of the segmentation unit, and stopping merging; and sending the merged basic data blocks and performing the operation processing.