WORM Storage Auto-Commit Detection via Time Range Counters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing WORM storage systems face inefficiencies in detecting auto-committed files and calculating the maximal expiration time of volumes, leading to performance penalties and delayed deletion of committed files due to the need for periodic scans.
Innovation Solution
Implementing a system of counters that track file changes over time ranges, allowing for real-time detection of auto-committed files and calculation of the maximal expiration time, enabling efficient management of WORM volumes without the overhead of repeated scans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic scans are performed to detect auto-committed files and calculate maximal expiration time, then the storage system can identify committed files for deletion, but the system suffers from performance penalties and delayed deletion due to the overhead of repeated scans
Solution Approach 1:
The system pre-calculates and stores the ctime (change time) attribute for each file when it is created or modified. This preliminary action eliminates the need for periodic scans to determine when files became immutable, as the system already has the timestamp information readily available to calculate auto-commit status and maximal expiration time.
Solution Approach 2:
Instead of scanning actual files to detect auto-committed status, the system uses a copy of time information (ctime attribute) that is stored separately in the file system metadata. This allows the system to determine auto-commit status by comparing stored timestamps rather than examining file contents or properties repeatedly.
2Reliability
If periodic scans are performed to verify no auto-committed files exist before volume deletion, then WORM compliance is maintained, but the discovery time of auto-committed files is delayed and administrative overhead increases
Solution Approach 1:
The system pre-calculates the maximal expiration time for volumes by using the stored ctime attributes of files. This preliminary calculation of expiration timelines allows the system to immediately determine when volumes can be deleted without waiting for periodic scans to discover auto-committed files.
Solution Approach 2:
The system continuously monitors file ctime attributes and automatically updates the maximal expiration time for volumes based on this feedback. When files are created, modified, or deleted, the system receives feedback about the change and recalculates the volume's expiration timeline, ensuring real-time WORM compliance without manual intervention or periodic scanning.
3Measurement precision
If all files in a volume are scanned to identify auto-committed files, then the maximal expiration time can be calculated accurately, but the overhead of repeated scans reduces system efficiency
Solution Approach 1:
The system uses stored ctime attribute copies instead of scanning actual files to determine auto-commit status. This allows accurate calculation of maximal expiration time by comparing stored timestamps without the performance overhead of repeated file system scans.
Solution Approach 2:
The file system automatically maintains and updates the ctime attributes for all files as part of its normal operation. This self-service approach ensures that the time information needed for maximal expiration time calculation is already available in the system without requiring separate scanning operations.
Data Source
AI summary
A method and a system for detecting an occurrence of an auto-commit operation applied to files managed by a file server compliant with write-once-read-many (WORM) rules. The method includes: allocating a plurality of non-overlapping predefined time ranges starting from a newest-changed-files time range and ending at an oldest-changed-files time range, wherein the time ranges add up to an auto-commit period associated with the auto-commit operation; repeatedly updating a count of files whose file-change-time is associated respectively with one of the allocated time ranges, wherein the updating is carried out every time the predefined time range lapses; and detecting, every time the time range lapses, an occurrence of an auto-commit operation applied to at least one of the files stored on the volume within a duration of the time range since the updating, whenever the count of the files associated with the oldest-changed-files time range is non-zero.