Backup Metadata Cache Generation for Faster Filesystem Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accessing metadata stored in backups consumes significant processing time and networking bandwidth due to the need to retrieve all data and metadata blocks from block-based storage devices.
Innovation Solution
A computing device generates a local metadata cache by using a scanner and extractor to identify and populate only the metadata blocks of a filesystem in a backup, reducing the need to read data blocks, thereby optimizing processing time and bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all data and metadata blocks are retrieved from block-based storage devices to access metadata in backups, then complete metadata access is achieved, but processing time and networking bandwidth are significantly consumed
Solution Approach 1:
The patent extracts only the necessary metadata blocks from the backup data, separating them from the unnecessary data blocks. This is achieved through scanning the backup index to identify metadata block locations and selectively retrieving only those blocks, thereby eliminating the need to process all data blocks and significantly reducing processing time while maintaining complete metadata access capability
Solution Approach 2:
The backup data structure is segmented into distinct data blocks and metadata blocks, with the index providing segmentation information. This allows the system to treat metadata blocks as a separate, independently accessible component, enabling selective retrieval of metadata without processing the entire backup dataset, thus resolving the time consumption issue
2Measurement precision
If all data and metadata blocks are retrieved from block-based storage devices to access metadata in backups, then complete metadata access is achieved, but networking bandwidth is significantly consumed
Solution Approach 1:
The system extracts only the essential metadata blocks from the backup through index-based location identification and selective retrieval. This extraction process filters out unnecessary data blocks, resulting in minimal network bandwidth consumption while ensuring all required metadata is obtained for complete access capability
Solution Approach 2:
Instead of retrieving all blocks (excessive action), the system performs partial action by retrieving only the specific metadata blocks needed. The index enables precise identification of metadata block locations, allowing the system to obtain exactly what is necessary without over-fetching data, thus optimizing bandwidth usage
3Productivity
If a local metadata cache is generated by identifying and populating only metadata blocks, then processing time and bandwidth usage are reduced, but additional storage space is required for the cache
Solution Approach 1:
The system performs preliminary action by pre-generating and storing a local metadata cache containing all metadata blocks before they are needed for access operations. This advance preparation eliminates the need to retrieve metadata from backup during actual access operations, significantly improving productivity. The cache size is limited to only metadata blocks, minimizing the storage space requirement compared to caching entire backup data
Data Source
AI summary
Example implementations relate to computer data storage. In some examples, a metadata scanner identifies files in a filesystem, wherein each file comprises logical blocks, and where the filesystem is included in a backup. The metadata scanner issues a read call for a logical block. A filesystem layer translates the read call into a set of translated read calls. For each translated read call, a metadata extractor determines whether the translated read call is to read a metadata block. In response to a determination that the translated read call is to read the metadata block, the metadata extractor obtains the metadata block from a persistent storage device, and stores the obtained metadata block in a metadata cache.