Consistent Hashing for Serverless Peer-to-Peer File Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing peer-to-peer file sharing architectures in distributed cloud storage systems are limited by reliance on a centralized server for coordination, leading to lower performance, scalability issues, and potential single points of failure, which increase costs and reduce reliability.
Innovation Solution
A peer-to-peer file sharing system utilizing a hash-based address space to distribute data chunks and ownership data among storage nodes, eliminating the need for a centralized server by using consistent hashing to determine adjacent nodes for data storage and retrieval, and implementing secure message authentication codes for verification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized server is used for coordination among storage nodes, then file ownership data and security features can be maintained, but system performance decreases and scalability becomes problematic
Solution Approach 1:
The patent extracts the centralized server function and distributes ownership data management to individual storage nodes. Each node maintains a local copy of the file ownership database, eliminating the single centralized coordination point while preserving the ability to manage file ownership, access keys, and security features distributed across the network.
Solution Approach 2:
The centralized file ownership database is segmented into multiple distributed copies held by different storage nodes. This segmentation allows the system to maintain file ownership information without requiring a single centralized server, improving performance by reducing coordination overhead while preserving reliability through redundant data distribution.
2Reliability
If a centralized server is used for coordination, then authentication and security features can be managed, but a single point of failure is created
Solution Approach 1:
The patent removes the centralized authentication server by embedding authentication capabilities directly into individual storage nodes. Each node performs local authentication decisions using stored security data, eliminating the single point of failure while maintaining secure file access through distributed verification of ownership and access rights.
Solution Approach 2:
Storage nodes perform self-service authentication and security verification using locally stored ownership data and access keys. This self-service approach eliminates dependency on a centralized authentication server, allowing nodes to independently verify file ownership and access permissions without creating a single point of failure.
3Reliability
If a centralized server is used for file transfer coordination, then file ownership verification can be performed, but all file transfers must route through the server
Solution Approach 1:
The patent extracts the file transfer coordination function from a centralized server and implements direct peer-to-peer transfers between storage nodes. Ownership verification is performed locally at the destination node using its stored file ownership database, eliminating the need for all transfers to route through a central server and reducing network overhead and energy consumption.
Solution Approach 2:
The patent introduces a distributed intermediary mechanism where each storage node acts as a local mediator for file transfers involving itself. Nodes use their locally stored ownership data as an intermediary verification mechanism, allowing direct transfers without requiring a centralized coordination server, thus improving transfer efficiency while maintaining verification reliability.
4Quantity of substance
If the number of storage nodes increases, then storage capacity and redundancy improve, but the centralized server storage and processing requirements increase linearly
Solution Approach 1:
The patent segments the centralized server processing function across multiple storage nodes. Each node handles file operations for files it owns or is responsible for, distributing the processing load. This segmentation allows the system to scale storage capacity by adding nodes without causing linear increase in any single server's processing requirements, as work is divided across the distributed network.
Solution Approach 2:
The patent applies local quality by having each storage node perform processing operations locally for files it manages, rather than centralizing all processing. Each node uses its locally stored ownership data to verify and process file access requests, enabling efficient local operations that scale with the number of nodes without requiring proportional increase in centralized processing capacity.
Data Source
AI summary
Systems, methods, and network attached storage nodes for peer-to-peer file sharing using consistent hashing for distributing data among storage nodes. A plurality of network attached storage nodes may be interconnected by a network and configured for peer-to-peer communication without a centralized server. When a node receives a user data file, it divides the file into data chunks and determines hashes for those data chunks. It uses consistent hashing of the chunk hashes and storage node addresses to map the data chunks to other nodes for distributed storage.