Logical Container Data Packing for File System Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file systems suffer from internal block fragmentation, leading to inefficient use of storage space and increased metadata overhead when storing large numbers of small data objects, as each object requires a separate container, resulting in wasted space and increased directory lookup stress.
Innovation Solution
The method involves packing multiple data objects into a single logical container, dividing blocks into extents to store data objects efficiently, and using metadata maps to track object locations, allowing for flexible storage and access of data objects within containers, thereby reducing unused space and metadata overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If each data object is stored in a separate container, then data access is simple and straightforward, but storage space is wasted due to internal block fragmentation and metadata overhead increases
Solution Approach 1:
The patent merges multiple data objects into a single container, allowing multiple objects to share the same storage space. This eliminates the need for separate containers for each small object, thereby reducing storage space waste while maintaining the ability to access individual objects through metadata maps.
Solution Approach 2:
The patent introduces metadata maps as an intermediary layer between the data objects and the storage container. These maps track the locations of data objects within the container, enabling simple access to individual objects while allowing multiple objects to share the same container space efficiently.
2Reliability
If each data object is stored in a separate container, then object isolation is maintained, but directory lookup stress increases and file system limits are strained
Solution Approach 1:
The patent combines multiple data objects into a single container, reducing the total number of containers that need to be managed. This decreases the stress on directory lookups and file system limits while maintaining object isolation through the use of metadata maps that track individual object locations within the shared container.
Solution Approach 2:
The metadata maps serve as an intermediary that maintains object isolation information within the shared container. These maps allow the system to track and access individual objects without requiring separate container entries, thereby reducing directory lookup complexity while preserving object isolation.
3Loss of substance
If block size is increased to reduce fragmentation, then storage efficiency improves, but the ability to store small objects efficiently deteriorates
Solution Approach 1:
The patent segments the storage block into multiple extents, allowing flexible allocation of storage space. This segmentation enables efficient storage of small objects while maintaining the ability to handle larger objects, thereby improving storage efficiency without sacrificing adaptability to objects of varying sizes.
Solution Approach 2:
The patent introduces dynamic extent allocation within blocks, allowing the storage system to adaptively allocate space based on object sizes. This dynamic approach enables efficient storage of small objects while maintaining flexibility for larger objects, resolving the contradiction between storage efficiency and adaptability.
Data Source
AI summary
An apparatus and method for containerization of multiple data objects within a block of a single container. The apparatus and method may pack multiple data objects together in a block of a logical container in a file system. The method may include receiving data in the form of multiple data objects to be stored in a file system, and collectively data packing the multiple data objects together in at least one block of a logical container in the file system. The block is a fundamental unit of storage space of the file system, and each block of the logical container includes multiple extents to store data from at least one data object of the multiple objects. The apparatus may include a plurality of storage devices coupled to a storage server. The storage server is configured to store the multiple data objects in at least one block of the logical container in the file system. Some data objects may be stored in multiple extents of one or more blocks depending on the size of the data object.