Online Data Consistency Checking in Network Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network storage systems require offline data consistency checking, which leads to prolonged downtime and high RAM usage, as they cannot approve or disapprove remedial changes before committing them to the file system.
Innovation Solution
Implementing a technique that allows users to approve or disapprove remedial changes by writing buffered data blocks to a change log file outside the volume, bypassing the usual consistency point process, enabling online data consistency checking and fixing without taking the entire storage server offline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If offline data consistency checking is performed using traditional tools, then data errors can be detected and fixed, but the storage server must remain offline causing prolonged downtime
Solution Approach 1:
The system performs preliminary data consistency checking and identifies remedial changes while the storage server is still online. The changes are prepared and staged before actual commitment, allowing the server to remain operational during the analysis phase. This preliminary action enables the switching to online checking mode without immediate service interruption.
Solution Approach 2:
The patent introduces an intermediary mechanism that buffers and manages remedial changes before they are committed to the file system. This intermediary layer allows the checking process to operate independently from the online service operations, enabling data verification without forcing the server offline.
2Reliability
If traditional offline checking tools are used, then comprehensive data verification can be performed, but a very large amount of RAM is consumed
Solution Approach 1:
The data checking process is segmented into multiple phases: online identification phase, offline commitment phase, and incremental application phase. During the online phase, only minimal data structures are loaded into RAM to identify issues. The actual remedial changes are committed incrementally during offline phases, reducing peak memory requirements while maintaining verification completeness.
Solution Approach 2:
The system performs partial checking online with reduced memory footprint, identifying issues without loading entire data structures into RAM. The remaining verification and commitment processes are distributed across multiple offline phases, achieving comprehensive verification without requiring excessive memory resources at any single point in time.
3Productivity
If remedial changes are automatically committed without user approval, then the checking process is faster, but users lose control over changes made to the file system
Solution Approach 1:
The system dynamically adjusts the commitment process based on user preferences and system state. Remedial changes are identified quickly during online operation, but the actual commitment is made dynamic - either automatic or manual - allowing the system to optimize for speed when appropriate while providing user control when needed. This dynamic approach resolves the contradiction between checking speed and user control.
Solution Approach 2:
The system implements feedback mechanisms where users can review identified issues and approved changes before commitment. The feedback loop allows users to control the change process while the system maintains high productivity by pre-identifying and preparing remedial actions. Users receive information about detected issues and can approve or modify the proposed changes before they are committed to the file system.
Data Source
AI summary
A network storage server includes a tool for detecting and fixing errors while the network storage server remains online (available for servicing client requests), which includes enabling a user to approve or disapprove remedial changes before the changes are committed. The technique bypasses the usual consistency point process for new or modified data blocks representing potential remedial changes. At a consistency point, dirty data blocks representing the potential remedial changes are written to a change log file residing outside the volume. The modified data blocks are written in sequential order to logical blocks of the change log file. In response to a user input indicating that a potential change should be committed, the corresponding modified data blocks are read from the change log file in the order in which they were written to the change log file, and they are written to persistent storage in that order.