Distributed Page Table for Storage Metadata Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for managing metadata in object storage systems face inefficiencies in access speed and storage space utilization, as they often rely on low-access-speed storage disks or require large memory spaces to improve metadata access efficiency.
Innovation Solution
A method that stores identifiers and page addresses as key-value pairs in a page table distributed across storage devices with varying access speeds, allowing for efficient metadata access while optimizing storage space by migrating blocks based on access frequency and available capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If metadata is stored on storage disks with low access speed, then storage space is optimized, but metadata access efficiency deteriorates
Solution Approach 1:
The patent divides the storage system into multiple storage devices with different access speeds (e.g., high-speed SSDs and low-speed HDDs). Metadata is segmented and distributed across these devices based on access frequency, with frequently accessed metadata stored on high-speed devices and less frequently accessed metadata stored on low-speed devices. This segmentation resolves the contradiction by allowing the system to optimize storage space using low-speed devices while maintaining fast access for critical metadata through high-speed devices.
Solution Approach 2:
The patent implements local quality by assigning different storage characteristics to different portions of metadata based on their access patterns. Hot metadata (frequently accessed) is stored on high-speed storage devices, while cold metadata (rarely accessed) is stored on low-speed storage devices. The page table mechanism enables the system to dynamically determine the storage location of each metadata entry, ensuring that each piece of metadata resides in the most appropriate storage location according to its access requirements, thus resolving the speed-storage space contradiction.
2Speed
If large memory space is allocated to improve metadata access efficiency, then access speed improves, but storage space consumption increases
Solution Approach 1:
The patent employs dynamic block migration between storage devices based on access frequency and available capacity. The page table tracks metadata access patterns and dynamically relocates metadata blocks between high-speed and low-speed storage devices. This dynamic adjustment allows the system to adapt to changing access patterns without requiring large fixed memory allocations, resolving the contradiction between access speed and memory space consumption by optimizing the balance dynamically rather than statically.
Solution Approach 2:
The patent changes the storage location parameter of metadata blocks based on access frequency and storage device capacity. The system monitors access patterns and modifies the physical location of metadata in the storage hierarchy, transitioning blocks between high-speed and low-speed devices as needed. This parameter change approach allows the system to maintain fast access performance while using minimal memory space, as metadata is placed on high-speed devices only when necessary based on current access requirements and available capacity.
3Quantity of substance
If metadata is distributed across multiple storage devices, then storage space utilization improves, but system complexity increases
Solution Approach 1:
The patent introduces a page table as an intermediary data structure that manages metadata distribution across multiple storage devices. The page table stores mappings between virtual metadata addresses and physical storage locations, abstracting the complexity of distributed storage management. This intermediary mechanism allows the system to utilize multiple storage devices effectively while hiding the complexity from the application layer, as the page table automatically handles the mapping and relocation operations, thus resolving the contradiction between storage space utilization and system complexity.
Data Source
AI summary
Metadata of a storage object is managed. An example method for managing metadata of a storage object comprises: in response to receiving a request to access metadata of a storage object, obtaining a first identifier of the storage object from the request; determining, based on the first identifier, a second identifier of a page where the metadata is located, an address of the page and the second identifier being stored in association in an entry of a page table distributed among a plurality of storage devices with different access speeds; determining, from the plurality of storage devices, a storage device storing the entry; and obtaining the address of the page by accessing the entry from the storage device, so as to access the metadata from the page.