Parallel File and Folder Replication for NAS Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file replication methods in network attached storage (NAS) systems are inefficient, leading to high processing times and increased load, which can result in replication failures, especially when dealing with large numbers of files.
Innovation Solution
A storage system that processes file replication folder by folder, executing file replication in parallel with folder replication, and periodically updating folder replication processing to reduce load and ensure high-speed, secure file replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If file replication is performed file by file, then replication accuracy is improved, but replication time and processing load increase significantly
Solution Approach 1:
The patent segments the replication process into two distinct modes: file-level replication for individual file changes and folder-level replication for bulk operations. This segmentation allows the system to apply the appropriate replication granularity based on the operation type, achieving both accuracy for individual files and efficiency for large-scale operations.
Solution Approach 2:
The system dynamically switches between file-level and folder-level replication based on real-time conditions. When file changes are detected during folder replication, the system adaptively adjusts the replication strategy to maintain data consistency without permanently committing to one approach, thus balancing accuracy and speed.
2Reliability
If file replication is performed file by file, then data consistency is improved, but processing load increases causing replication failure
Solution Approach 1:
The patent divides the replication workload into file-level tasks and folder-level tasks. Folder-level replication handles bulk operations with lower processing overhead, while file-level replication manages individual file consistency. This segmentation prevents the processing load from becoming overwhelming while maintaining data consistency through coordinated execution of both replication modes.
Solution Approach 2:
The system introduces an intermediary mechanism that coordinates between folder-level and file-level replication processes. This intermediary manages the interaction between the two replication modes, ensuring that file-level changes are properly integrated into folder-level operations without creating excessive processing conflicts or data inconsistency.
3Productivity
If folder replication is executed, then replication speed is improved, but individual file changes may be missed
Solution Approach 1:
The system implements feedback mechanisms that monitor file changes during folder replication operations. When individual file modifications are detected while folder-level replication is in progress, the system receives feedback about these changes and adjusts the replication process accordingly, ensuring that no file changes are missed while maintaining the overall speed advantage of folder-level replication.
Solution Approach 2:
The patent ensures continuous replication action by running both folder-level and file-level replication processes in coordination. Rather than stopping folder replication to handle individual file changes, the system maintains continuous operation of both replication modes simultaneously, ensuring that no file changes are missed while preserving the high speed of folder-level replication.
Data Source
AI summary
A storage system includes processing circuitry configured to detect a file operation in a folder of a first storage, sequentially register information of files having undergone a change as a file replication processing target in a database, execute file replication processing to sequentially replicate each individual file of the files registered as the file replication processing target to a second storage, execute folder replication processing to replicate the folder to the second storage, in the folder replication processing, exclude a file belonging to the folder for the folder replication processing from the file replication processing target, and execute the file replication processing in parallel with the folder replication processing when the information of the file having undergone a change has been registered as the file replication processing target in the database during the folder replication processing.