Hierarchical Storage Management with Data Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data synchronization and hierarchical storage management solutions are disjointed, leading to inefficiencies in data access and storage costs, as they fail to provide reliable read/write access to cold storage tiers while maintaining data coherence across nodes.
Innovation Solution
A system that configures a processor to host a file system representing a replica of file systems on multiple nodes, using a slower second storage medium for less frequently accessed data, with mechanisms for creating immutable snapshots, detecting changes, and ensuring data coherence through synchronization and locking mechanisms, allowing read/write access to cold storage while preventing conflicts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data synchronization solutions store data on multiple nodes for convenient access, then data accessibility is improved, but storage cost increases because each node must store the complete data
Solution Approach 1:
The system divides data into two segments: a small metadata portion stored on each node's fast storage medium for quick access, and a large data content portion stored on a centralized slow storage medium. This segmentation allows nodes to have fast access to metadata while avoiding the cost of storing complete data copies on each node.
Solution Approach 2:
The patent introduces a centralized storage medium as an intermediary that holds the actual data content. Nodes access this intermediary through a file server, eliminating the need for each node to maintain full data copies while still providing convenient data access across the distributed system.
2Quantity of substance
If hierarchical storage management puts less frequently used data on slower cold storage media, then storage cost is reduced, but data access speed decreases
Solution Approach 1:
The system applies different storage quality characteristics to different data portions: fast storage media is used locally at each node for metadata that needs quick access, while slow storage media is used centrally for data content that is less frequently accessed. This local quality differentiation optimizes both cost and performance according to actual access patterns.
Solution Approach 2:
Instead of storing complete data copies on each node (excessive action), the system stores only the necessary metadata locally (partial action) and retrieves data content from centralized storage only when needed, achieving cost savings while maintaining acceptable access speeds for most operations.
3Ease of operation
If the system provides read/write access to cold storage tiers, then data accessibility is improved, but data coherence and reliability across nodes becomes difficult to maintain
Solution Approach 1:
The system implements feedback mechanisms where the file server monitors access patterns and updates metadata on cold storage tiers accordingly. When data is accessed, the system feedbacks this information to update access timestamps and coherence states, ensuring data consistency across nodes while maintaining cost-effective storage on slower media.
Solution Approach 2:
The system performs preliminary actions by pre-loading frequently accessed data and updating metadata coherence states before actual data access operations. This ensures that when nodes need to access data from cold storage, the coherence information is already prepared, maintaining reliability without requiring expensive fast storage on all nodes.
Data Source
AI summary
A system hosts a file system representing a replica of file systems hosted on a plurality of nodes, each node storing data on a first storage medium. The system stores data on a second storage medium that is slower than the first storage medium. The system stores on the second storage medium a file received from one of the nodes when the file remains un-accessed by the nodes for a predetermined period of time. The first storage medium on the plurality of nodes stores a pointer to the file and not the data content of the file stored on the second storage medium. The system provides the file from the second storage medium to the first storage medium on one of the nodes while providing the plurality of nodes read and write access to the file stored on the second storage medium using the pointer on the first storage medium.