File-Specific Data Scrubbing via Distributed Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face issues with data loss and corruption due to long periods of idleness, where block-level scrubbing operations can only report Logical Block Addresses (LBAs) of faulty data, but not efficiently translate these to file displacements, leading to increased processing delays and reduced system efficiency.
Innovation Solution
A method that involves scanning blocks for identifiers and using metadata in trailers to identify additional blocks belonging to the same file, allowing for file-specific data scrubbing and reporting, thereby enabling the identification of impacted files and facilitating targeted recovery actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If block-level scrubbing operations are performed to detect drive failure, then data loss can be prevented, but processing delays increase and system efficiency decreases due to inability to translate LBAs to file displacements
Solution Approach 1:
The patent segments the metadata storage by creating separate trailers at the end of each data block, rather than using a centralized metadata structure. Each trailer contains file identification information specific to its block, enabling independent processing of block-level scrubbing operations without requiring centralized metadata lookups, thus improving system efficiency while maintaining reliability
Solution Approach 2:
The patent introduces trailers as intermediary structures between data blocks and the filesystem layer. These trailers contain file identification information that acts as a mediator, enabling direct translation from LBA to file identity without requiring complex centralized metadata operations, thereby reducing processing delays while preventing data loss
2Ease of operation
If centralized inode list is used to track file blocks, then file block locations can be managed, but processing complexity increases and translation efficiency decreases
Solution Approach 1:
The patent extracts file identification information from the centralized inode list and places it directly in trailers at the end of each data block. This extraction eliminates the need for complex centralized metadata operations during scrubbing, as each block carries its own file identification information, thereby reducing processing complexity while maintaining ease of file block tracking
Solution Approach 2:
Each data block becomes self-sufficient by containing its own file identification information in the trailer. This self-service approach eliminates dependency on centralized metadata structures for basic file identification operations, reducing processing complexity while maintaining the ability to track file blocks effectively
Data Source
AI summary
A computer-implemented method, according to one embodiment, includes: determining whether an error has occurred for each block of storage space in memory by sequentially evaluating each of the blocks of storage space. In response to determining that an error has occurred for a given block, the given block is scanned for an identifier at an end of data stored in the given block, and a trailer appended to the identifier is located. Metadata included in the trailer is further used to identify each additional block which corresponds to a same file as the given block, and a data scrubbing procedure is performed on the given block and each of the additional identified blocks. A result of the scrubbing procedure is also output in response to detecting an error in the given block and/or the additional identified blocks.