Data Flushing Maturity Level Sorting for Disk Array Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data flushing strategies for DRAM cache, such as time-sorting and LBA-sorting, face inefficiencies in disk access and resource utilization, particularly in disk arrays where data is scattered across multiple physical disks, leading to reduced bandwidth utilization and increased resource occupation.
Innovation Solution
A method where storage segments are inserted into lists based on maturity levels, indicating the proportion of data-written blocks, and flushed in descending order of maturity levels to prioritize high-maturity segments, optimizing data writing to disk stripes and improving bandwidth utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is flushed to disk array using traditional time-sorting or LBA-sorting strategies, then data is written to disk, but bandwidth utilization is reduced and resource occupation increases due to scattered data distribution across multiple physical disks
Solution Approach 1:
The patent segments the flushing process by introducing maturity levels that categorize storage segments based on their data completeness. Storage segments are divided into different maturity levels (e.g., partially filled vs. fully filled), and each level is flushed separately. This segmentation allows the system to optimize flushing operations by handling high-maturity segments that can be written more efficiently to disk stripes, thereby improving bandwidth utilization while reducing overall resource occupation.
Solution Approach 2:
The patent changes the parameter used for flushing priority from traditional time-based or LBA-based sorting to maturity level-based sorting. By introducing maturity level as a new parameter that reflects the proportion of data-written blocks to total blocks in a storage segment, the system can prioritize flushing segments with higher maturity levels. This parameter change enables more data to be written to each stripe in a single flush operation, improving bandwidth utilization and reducing the number of flush operations required.
2Quantity of substance
If data is scattered across multiple physical disks in disk array, then storage capacity is increased, but bandwidth utilization is reduced due to increased number of access points
Solution Approach 1:
The patent merges multiple storage segments with high maturity levels into a single flush operation. By identifying and combining segments that are ready to be flushed (high maturity level), the system can write more data to each disk stripe in a single operation, thereby improving bandwidth utilization. This merging approach allows the system to take advantage of the parallel storage capacity of multiple physical disks while minimizing the overhead of multiple separate access operations.
3Reliability
If traditional flushing strategies are used, then data is flushed to disk, but resource occupation increases due to multiple flush operations
Solution Approach 1:
The patent performs preliminary classification of storage segments into different maturity levels before the flushing operation. By pre-categorizing segments based on their data completeness, the system prepares the flushing queue in advance, allowing high-maturity segments to be prioritized and batched together. This preliminary action reduces the need for multiple separate flush operations, thereby reducing resource occupation while ensuring complete data flushing.
Solution Approach 2:
The patent implements a feedback mechanism where the maturity level of storage segments is continuously monitored and used to adjust flushing priorities. The system tracks the proportion of data-written blocks to total blocks in each storage segment and uses this feedback information to dynamically prioritize which segments to flush first. This feedback-driven approach ensures that data is flushed completely and efficiently, reducing the number of re-flush operations needed and thereby reducing resource occupation.
Data Source
AI summary
In techniques for flushing data, based on a maturity level of a storage segment, the storage segment is inserted into a list to be flushed corresponding to the maturity level in a plurality of lists to be flushed, the plurality of lists to be flushed respectively correspond to different maturity levels, and the maturity level at least indicates a proportion of the number of data-written blocks to the total number of blocks of the storage segment; and the list to be flushed for the corresponding maturity level in the plurality of lists to be flushed is flushed to a disk array according to a descending order of the maturity levels. In this way, the bandwidth utilization of the disk array can be improved.