Distributed File Cache Consistency via Blockchain and DHT
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional distributed file cache systems face limitations in data sharing among nodes, lack data consistency during updates, and do not consider distance for data access, leading to inefficient file retrieval and potential inconsistencies.
Innovation Solution
A method that utilizes a distributed hash table to record contact information for file access and a blockchain to manage file versions, ensuring data consistency and enabling fine-grained data sharing by generating and recording contact and version information, allowing nearby file access and reuse of unchanged file slices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a traditional distributed file cache system is used, then file storage and retrieval are enabled, but data sharing among nodes is limited and data consistency during updates cannot be ensured
Solution Approach 1:
The patent introduces a blockchain as an intermediary mechanism to manage file version information and a distributed hash table (DHT) as an intermediary to manage contact information. These intermediaries enable reliable data sharing and consistency without requiring complex centralized control, resolving the contradiction between reliability and system complexity.
Solution Approach 2:
The patent segments the file management system into distinct functional components: a blockchain for version management, a DHT for contact information storage, and individual node caches for local file storage. This segmentation allows each component to specialize in specific tasks, improving overall reliability while keeping individual components relatively simple.
2Productivity
If centralized file cache management is used, then data consistency can be maintained, but access efficiency and scalability are reduced
Solution Approach 1:
The patent implements self-service mechanisms where nodes autonomously manage their own cache, independently track file versions through the blockchain, and automatically update their contact information in the DHT. This eliminates the need for centralized management while maintaining data consistency through distributed cooperation, thereby improving both productivity and reliability.
Solution Approach 2:
The system incorporates feedback mechanisms where nodes continuously update the blockchain with version information and the DHT with contact information. This feedback loop ensures that all nodes have up-to-date information about file locations and versions, enabling efficient retrieval while maintaining consistency without centralized control.
3Reliability
If all file data is stored at every node, then data availability is maximized, but storage resources are wasted on redundant data
Solution Approach 1:
The patent applies local quality by enabling each node to store and cache only the specific file data it needs based on its local requests and the distributed hash table contact information. Nodes maintain local copies of files they frequently access while the blockchain and DHT provide centralized coordination, achieving high data availability without requiring every node to store every file, thus optimizing storage resource usage.
Data Source
AI summary
Embodiments of the present invention relate to methods, systems, and computer program products for file management in a distributed file cache system. In some embodiments, a method is disclosed. According to the method, responsive to determining that at least one client node is obtaining a file of a first version stored at a storage node, one or more processors generate contact information indicating that the file of the first version is accessible from the storage node and the at least one client node and recorded the contact information into a distributed hash table. The storage node and the at least one client node are included in a plurality of nodes associated with the distributed hash table. Further, one or more processors generate first version information indicating that the file is of the first version and record the first version information into a blockchain associated with the plurality of nodes.