Data Mover Recall Size Analysis for Storage Protocol Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management methods face inefficiencies in data transfer between file servers and remote storage systems, particularly when accessing large files, leading to burdens on communication bandwidth and file server capacity, due to mismatches in data transfer units between protocols.
Innovation Solution
A computer system with a control unit, data mover, log obtaining unit, and recall size analysis unit that analyzes access characteristics to determine optimal recall sizes and performs stubification processes to manage data transfer efficiently between local and remote storage systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transferred in small units (64 Kbytes) to match file sharing protocol requirements, then compatibility with NFS/CIFS is maintained, but data transfer efficiency deteriorates
Solution Approach 1:
The patent segments data transfer into two distinct phases: (1) bulk data transfer from remote storage to file server in large units optimized for storage protocols, and (2) file sharing operations from file server to client in small units compatible with NFS/CIFS protocols. This segmentation allows each phase to use optimal transfer units for its specific purpose, resolving the contradiction between protocol compatibility and transfer efficiency.
Solution Approach 2:
The file server acts as an intermediary between the remote storage system and the client. It receives data in large efficient units from remote storage, then serves clients in small protocol-appropriate units. This intermediary role allows the system to decouple the transfer unit requirements of remote storage protocols from file sharing protocols, achieving both efficiency and compatibility.
2Speed
If entire large files are read from remote storage system, then application performance is improved, but communication bandwidth and file server capacity are burdened
Solution Approach 1:
The system performs preliminary analysis of access patterns using logs to determine recall sizes before actual data transfer. By pre-calculating optimal recall sizes based on historical access characteristics, the system prepares appropriate data quantities in advance, avoiding unnecessary transfer of excessive data and thus reducing bandwidth consumption while maintaining application performance.
Solution Approach 2:
The system implements feedback mechanisms by analyzing access logs and using the results to dynamically adjust recall sizes. The log analysis unit continuously monitors access patterns and feeds this information back to the data mover, which then optimizes the amount of data recalled from remote storage. This closed-loop feedback ensures that the right amount of data is transferred to satisfy application performance requirements without wasting bandwidth.
Data Source
AI summary
A computer system is coupled to a local storage system and a remote storage system, and comprises a data mover, and a recall size analysis unit. The recall size analysis unit uses a combination of type of the data and size of the data as a retrieval key of an access characteristics to analyze the logs of the same access characteristics and to determine a recall size for each of the access characteristics. The data mover obtains a target data corresponding to the recall size of the access characteristics of the target data from the remote storage system in a case of receiving a read request for the target data that is not stored in the local storage system.