Distributed Storage Cache Management for Network Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in reducing network load and improving responsiveness due to high data transfer requirements and potential bottlenecks, especially during server failures or when data needs to be migrated between servers.
Innovation Solution
A distributed storage system with an in-charge storage apparatus that manages data by transferring and caching data from other storage devices, reducing network load by storing cache data locally and ensuring data availability during server failures or migrations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored distributed across multiple servers, then storage capacity and reliability are improved, but network load and data transfer requirements increase
Solution Approach 1:
The system performs preliminary actions by caching data from data areas to cache areas before actual read requests occur. This anticipatory caching reduces the need for frequent data transfers across the network when data is accessed, thereby reducing network load while maintaining distributed storage reliability.
Solution Approach 2:
The cache area acts as an intermediary between the data area and the read request processing. By introducing this intermediate storage layer, the system reduces direct data transfer requirements across the network, as data can be served from the local cache area rather than being repeatedly transferred from remote data areas.
2Speed
If data is cached in local storage devices, then access responsiveness is improved, but device complexity increases
Solution Approach 1:
The storage device is segmented into distinct functional areas: data areas for storing original data and cache areas for storing cached data. This segmentation allows each area to have specialized functions, improving access responsiveness for frequently accessed data while keeping the overall device structure manageable through clear functional separation.
3Productivity
If data is migrated between servers to balance load, then system performance is improved, but data transfer time and network load increase
Solution Approach 1:
The system performs preliminary caching of data that is likely to be accessed, rather than waiting for actual access patterns to emerge. This preliminary action reduces the need for time-consuming data migration operations later, as data is already positioned in optimal locations before access demands arise.
Solution Approach 2:
Instead of migrating entire data sets between servers, the system creates copies of frequently accessed data in cache areas of servers that will need them. This copying approach allows load balancing without the overhead of complete data migration, reducing both transfer time and network load.
Data Source
AI summary
Provided is a distributed storage system which can reduce a load on a network between storage apparatuses when an access request is received and improve responsiveness. In the distributed storage system, the storage device includes a data area and a cache area; a node becomes an owner node when receiving a transfer of charge of an LU from another node in a non-storage state where LU data is not stored in a data area; the processor of the owner node receives a read request for an LU that is in charge, obtains data of a target area based on data of the storage device of another node when the data of the target area is not stored in the data area or the cache area of the owner node, and transmits the data to a request source and stores it in a cache area.