Dynamic Storage Allocation for Fragmentation Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed file systems, fixed block sizes lead to internal and external fragmentation, resulting in inefficient storage device usage and decreased Input/Output (IO) access efficiency due to non-consecutive data arrangement and unused storage areas.
Innovation Solution
The system dynamically determines the storage device for writing data based on data size, increase ratio, and system load, using a control unit to set reference block sizes and values, and temporarily caching data when the system load is high, thereby optimizing storage allocation and reducing fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed block size is used for writing data to storage devices, then the writing speed is maintained, but internal fragmentation occurs resulting in inefficient storage device usage
Solution Approach 1:
The system dynamically selects between multiple storage devices with different block sizes based on the data size to be written. When data size is small, it selects a storage device with a smaller block size to reduce internal fragmentation. When data size is large, it selects a storage device with a larger block size to maintain writing speed. This dynamic selection resolves the contradiction between maintaining writing speed and reducing storage waste.
Solution Approach 2:
The system changes the block size parameter by selecting different storage devices with different block sizes according to the data size. This parameter change allows the system to optimize both writing speed and storage capacity utilization by matching the block size to the data size, thereby resolving the contradiction between these two parameters.
2Device complexity
If a fixed block size is used for writing data to storage devices, then the storage structure is simplified, but external fragmentation occurs resulting in decreased IO access efficiency
Solution Approach 1:
The system dynamically determines the optimal storage device by considering both the data size and the characteristics of multiple storage devices with different block sizes. This dynamic determination strategy prevents external fragmentation by selecting the most appropriate storage device for each writing operation, thereby maintaining high IO access efficiency without significantly increasing system complexity.
3Productivity
If data is written immediately to storage devices, then the writing speed is maintained, but storage devices with fixed block sizes cause fragmentation and reduced efficiency
Solution Approach 1:
The system performs preliminary action by determining the optimal storage device before actually writing the data. It calculates the data size, compares it with the block sizes of available storage devices, and selects the most appropriate storage device in advance. This preliminary determination ensures that data is written efficiently without causing fragmentation, thereby maintaining both writing speed and storage efficiency.
Data Source
AI summary
An information processing system comprises a processor. The processor executes a process that causes the information processing system to perform first writing, when requested to write data, data from which to subtract remainder data obtained by dividing a data size of write target data by a first data size of first processing unit data, the first data size being a size of processing unit data of a reading/writing process in a first storage device, in the write target data requested to be written to the first storage device and second writing the remainder data not being written by the first writing to a second storage device in which to set a second data size of second processing unit data, the second data size being smaller than the first data size of the first processing unit data.