Memory Record Ownership Negotiation in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional direct-attached storage (DAS) models are inefficient, inconvenient, and not resilient due to lack of resource sharing between servers, leading to data loss upon single server failure, while traditional shared storage architectures suffer from high network latency.
Innovation Solution
A method for negotiating memory record ownership between network nodes, using a placement algorithm to decide whether to relocate memory records to minimize network latency, by either providing remote access or relocating data to the requesting node, and optionally utilizing caching and third-party shared storage to manage data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored locally on each server using direct-attached storage (DAS), then access speed is improved, but resource sharing capability deteriorates
Solution Approach 1:
The system segments data storage across multiple servers, with each server holding portions of the file system. This allows local access to segmented data while enabling cross-server sharing through the distributed architecture, resolving the contradiction between fast local access and resource sharing capability
Solution Approach 2:
Each server in the distributed file system serves multiple functions: it acts as both a storage node for data segments and a access point for other servers. This multi-functionality enables servers to both maintain local data for fast access and share resources with the broader network, simultaneously achieving speed and adaptability
2Ease of operation
If data is stored on a single server, then data access simplicity is improved, but system reliability deteriorates
Solution Approach 1:
The file system is segmented and distributed across multiple servers, with each server storing specific portions of data. This segmentation maintains simple access within each server while providing redundancy across the distributed system, resolving the contradiction between access simplicity and reliability
Solution Approach 2:
Each server maintains local storage capacity and file system segments, providing simple local access while contributing to overall system reliability through distributed redundancy. The local quality of each node ensures both ease of operation and system-wide reliability
3Productivity
If data is uniformly distributed across all nodes, then load balancing is improved, but data access speed deteriorates
Solution Approach 1:
The system implements non-uniform data distribution where frequently accessed data and metadata are placed on local nodes, while less frequently accessed data is distributed across the network. This local quality approach maintains load balancing for overall system productivity while optimizing access speed for commonly used data
Solution Approach 2:
The system performs preliminary placement of data and metadata on appropriate nodes based on access patterns and application requirements. This preliminary action ensures that data is pre-positioned for optimal access speed while maintaining balanced load distribution across the distributed file system
Data Source
AI summary
A method of negotiating memory record ownership between network nodes, comprising: storing in a memory of a first network node a subset of a plurality of memory records and one of a plurality of file system segments of a file system mapping the memory records; receiving a request from a second network node to access a memory record of the memory records subset; identifying the memory record by using the file system segment; deciding, by a placement algorithm, whether to relocate the memory record, from the memory records subset to a second subset of the plurality of memory records stored in a memory of the second network node; when a relocation is not decided, providing remote access of the memory record via a network to the second network node; and when a relocation is decided, relocating the memory record via the network for management by the second network node.