Storage Server Constant-Time Copy via Snapshot References
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Copy operations in network storage servers take significant user-perceivable time for large files, disrupting user activities as they depend on the size of the source file system object, and existing technologies do not provide efficient handling of file system operations during the copying process.
Innovation Solution
A system and method that generates a snapshot of the source file system object before copying, allowing other file system operations during the copy process by referencing data blocks instead of copying them, and skipping unnecessary data copies if the destination object has been modified, thereby enabling truncate and append operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the storage server copies all contents of the source file system object before making it available, then data integrity is ensured, but the copy operation time becomes noticeable and interferes with user activities
Solution Approach 1:
The system performs preliminary actions by creating the destination file system object and generating a snapshot of the source object before the copy process begins. This allows the object to be available for operations before all data is copied, reducing user-perceivable wait time while maintaining data integrity through the snapshot reference mechanism.
Solution Approach 2:
The system introduces a snapshot as an intermediary mechanism that allows read operations during the copy process. The snapshot acts as a mediator between the ongoing copy operation and user access requirements, enabling users to access data without waiting for the complete copy while ensuring data consistency.
2Stability of the object's composition
If the system waits for the complete copy process before allowing other operations, then data consistency is maintained, but user productivity is reduced due to the large time interval
Solution Approach 1:
The system dynamically adjusts its behavior based on the copy process state. It allows read operations during copying by referencing the snapshot, permits write operations that modify the destination object, and skips copying for modified portions. This dynamic approach maintains data consistency while enabling continuous user productivity throughout the copy process.
Solution Approach 2:
The system enables continuous useful actions by allowing file system operations to proceed during the copy process. Users can perform reads, writes, truncates, and appends on the destination object without waiting for the copy to complete, maintaining productivity while the copy operation continues in the background with modified handling.
3Quantity of substance
If the system copies data blocks directly, then complete data replication is achieved, but the operation takes significant time for large files
Solution Approach 1:
The system uses copying at the metadata level rather than data level. It copies the buffer tree structure and creates references to data blocks instead of duplicating the actual data contents. This approach achieves complete data replication through reference copying, reducing copy time from hours to seconds while maintaining data integrity.
Solution Approach 2:
The system applies different quality levels to different parts of the copy process. Metadata and buffer tree structures are copied completely and immediately, while data blocks are referenced rather than duplicated. This local differentiation between metadata copying and data referencing achieves complete replication with minimal time loss.
4Ease of operation
If the system provides immediate access to the new object, then user accessibility is improved, but data may be accessed before complete copying occurs
Solution Approach 1:
The snapshot serves as an intermediary that guarantees data availability for read operations. When users access the destination object during the copy process, the system provides data from the snapshot, ensuring that complete and consistent data is available immediately without waiting for the copy to finish, thus maintaining both accessibility and reliability.
Data Source
AI summary
A system and method for providing a substantially constant-time copy operation for file system objects managed by a storage server begins by generating a snapshot of at least a portion of a data set managed by the storage server. The system then performs a copy operation in the storage server to generate a copy of the data set separate from the snapshot, on a set of block locations containing a predetermined reference value. During the copy operation to generate the copy of the data set separate from the snapshot, the system can receive from a requester a first read request directed to the copy of the data set that the copy operation is to generate. In response to the first data request, the system provides data from the snapshot to the requester by the storage server.