Distributed Storage Latency Reduction via Data Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems with a shared data set experience significant latency in read and write requests due to geographical distance and the need for remote sites to access data from a central owner site, leading to inefficiencies in data access and management.
Innovation Solution
Implementing synchronization and caching methods to reduce latency by periodically transmitting images of the shared data set to remote sites and using a migration method to transfer ownership of the shared data set to the site with the highest activity, ensuring seamless access and minimizing downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If remote sites access data from the central owner site, then data consistency is maintained, but access latency increases due to geographical distance
Solution Approach 1:
The patent implements preliminary action by proactively copying data from the owner site to remote sites before actual access requests occur. The system periodically transmits data images to remote sites and maintains local caches, so that when read requests arrive, data is already available locally rather than needing to be fetched from the distant owner site at that moment.
Solution Approach 2:
The patent applies copying by creating and transmitting data images from the owner site to remote sites. Instead of having all sites access the original data at the owner site, the system generates copies of the data and distributes them to remote locations, allowing local access without network latency while maintaining data consistency through periodic synchronization.
2Loss of time
If data is cached locally at remote sites, then read latency is reduced, but data may become stale and require synchronization overhead
Solution Approach 1:
The patent implements periodic action by establishing regular synchronization intervals where data images are transmitted from the owner site to remote sites. Instead of continuous real-time synchronization which would be complex and resource-intensive, the system uses periodic updates at predetermined intervals, balancing freshness of data with reduced synchronization overhead.
Solution Approach 2:
The patent uses data images as intermediaries between the owner site and remote sites. Rather than maintaining direct real-time connections for all data access operations, the system creates intermediate data representations that can be transmitted and cached independently, simplifying the synchronization mechanism while enabling local access.
3Reliability
If ownership of shared data set remains at the original site, then data integrity is maintained, but remote sites experience higher latency
Solution Approach 1:
The patent applies copying by transmitting data images from the owner site to remote sites, creating local replicas that preserve data integrity while enabling fast local access. The copied data maintains consistency with the original through periodic synchronization, resolving the conflict between maintaining data integrity at the original site and reducing access latency at remote sites.
Data Source
AI summary
Described are methods and apparatus for reducing latency of read and write requests for a set of storage system sites having a shared data set. An owner site may directly write to the shared data set and contains current data regarding the shared data set. The remote sites may experience substantial latency when accessing the shared data set stored at the owner site. Synchronizing and caching methods may reduce overall read latency experienced at remote sites by periodically transmitting images of the shared data set to the remote sites. Also, a migration method may be used to change ownership of the shared data set from a current owner site (that may be receiving a relatively low number of read/write requests) to a new owner site (that may be receiving a relatively high number of read/write requests) to reduce the overall read and write latency experienced in the sites.