Uncorrectable User Data Identification in RAID Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current consistency-checking tools in data storage systems, such as FSCK, are inadequate in identifying and addressing uncorrectable user data due to their inability to efficiently check user data integrity, particularly in RAID storage systems, leading to potential data loss and prolonged unawareness of errors.
Innovation Solution
Accessing RAID metadata to identify broken RAID arrays and marking metadata that points to uncorrectable user data, allowing for quick identification of user data on broken arrays without reading the data, and optionally performing selective reads to verify data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If FSCK checks all user data blocks for integrity, then data loss is reduced, but system downtime increases significantly
Solution Approach 1:
The patent segments the data checking process into two distinct phases: a fast metadata-only checking phase that can be performed frequently with minimal downtime, and a slower comprehensive user data checking phase that is performed less frequently. This segmentation allows the system to maintain data integrity while minimizing the impact on system availability by making the checking frequency and depth configurable.
Solution Approach 2:
The patent implements partial checking by allowing administrators to specify which portions of user data to check (e.g., only critical files, only certain directories, or random sampling). This partial action approach provides a middle ground between checking nothing and checking everything, enabling data integrity verification with reduced time overhead compared to full system checks.
2Measurement precision
If FSCK performs comprehensive user data checking, then uncorrectable data is identified, but checking speed decreases
Solution Approach 1:
The patent divides the checking process into hierarchical levels: Level 1 checks only metadata structures (fast, provides basic integrity assurance), while Level 2 checks extend to user data blocks (slower, provides comprehensive integrity verification). This segmentation allows the system to achieve high measurement precision when needed while maintaining high productivity for routine checks.
Solution Approach 2:
The patent enables partial user data checking where only specific portions of the filesystem are examined in detail. By allowing selective checking of user data rather than requiring complete examination, the system achieves sufficient error detection capability with significantly improved checking speed compared to exhaustive verification.
3Productivity
If FSCK checks metadata only, then checking time is reduced, but uncorrectable user data remains undetected
Solution Approach 1:
The patent performs preliminary metadata checking as a first step to quickly identify and isolate potential problems. By checking metadata structures first (which organize and reference user data), the system can rapidly detect corruption that would affect user data accessibility. This preliminary action provides early warning of integrity issues without requiring immediate comprehensive user data verification.
Solution Approach 2:
The patent uses metadata structures as an intermediary layer between the filesystem organization and actual user data. By checking this intermediary layer first, the system can infer the health status of user data without directly examining every data block. The metadata acts as a proxy that provides efficiency benefits while still enabling detection of many integrity problems.
Data Source
AI summary
A technique for managing user data in a storage system includes accessing RAID metadata to identify user data that the storage system backs with broken RAID arrays. The technique further includes marking metadata that points to at least some of that user data to identify such user data as uncorrectable.