Hash Memory Table for Cloud Storage Metadata Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional metadata accessing methods in cloud storage systems face inefficiencies due to low access locality, leading to increased input/output (I/O) operations and decreased performance, especially with update-intensive workloads.
Innovation Solution
A hash memory table is built with index and verification hash codes, loaded into a buffer using a bucket-based replacement policy, allowing for efficient file access by embedding logical file names into physical addresses and reducing disk I/O operations through hash-based file identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional database systems are used to manage metadata, then metadata can be organized and managed, but disk scheduling flexibility is limited and buffer memory usage is inefficient due to random operations
Solution Approach 1:
The patent segments the buffer memory into multiple buckets, each associated with a specific hash code range. This segmentation allows the system to directly access specific buckets based on hash codes of file keys, eliminating the need for random disk operations and improving disk scheduling flexibility while maintaining efficient metadata management.
2Quantity of substance
If the number of metadata increases, then more files can be managed, but I/O speed decreases and overall performance is significantly limited
Solution Approach 1:
The patent pre-computes and stores hash codes for all file keys in the hash memory table before access operations. When a file needs to be accessed, the system only needs to compute the hash code of the key and directly locate the corresponding bucket in the buffer, avoiding sequential searches and reducing I/O operations even as the number of metadata entries increases.
3Speed
If buffer memory is used to cache metadata, then access speed can be improved, but buffer memory is quickly exhausted with large numbers of files
Solution Approach 1:
The patent transitions from a traditional linear buffer structure to a multi-dimensional hash-based bucket structure. By organizing buffer space into multiple buckets indexed by hash codes, the system can efficiently utilize available buffer memory and directly access specific regions based on hash codes, improving access speed while accommodating larger numbers of files without exhausting buffer capacity.
Data Source
AI summary
A method for accessing files on a storage system is provided. A hash memory table including a plurality of hash buckets respectively corresponding to a plurality of index hash codes is built. Each of the hash buckets has a pointer pointing towards at least one entry. Each of the entries has a physical address field and a hash code field. The physical address fields respectively record physical addresses storing the files, and the hash code fields respectively record verification hash codes corresponding to the files. The index hash codes are generated by inputting keys of the files to an index hash function and the verification hash codes are generated by inputting keys of the files to a verification hash function. Then, the hash memory table is loaded into the buffer with a bucket-based replacement policy so that the files are able to be accessed according to the hash memory table.