Distributed Storage Web Services Keymap Namespace
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computing systems face limitations in data accessibility and reliability, particularly in accessing and storing application data across different devices and platforms, due to restrictive network access and inconsistent data redundancy features.
Innovation Solution
A distributed web-services based storage system is implemented, utilizing a web services interface, storage nodes, and a keymap instance to manage data objects with unique keys and locators, enabling replication and access across multiple nodes for enhanced accessibility and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored on local hard drive of a particular computer system, then data storage is simple and direct, but data accessibility to applications on other systems is limited
Solution Approach 1:
The storage system is segmented into multiple independent storage nodes distributed across different systems. Each node stores portions of data objects, allowing applications on any system to access data by contacting any node. This segmentation resolves the contradiction by distributing data across the network boundary while maintaining simple access through the namespace service.
Solution Approach 2:
A namespace service acts as an intermediary between applications and distributed storage nodes. The namespace service translates application requests into node-specific operations, shielding applications from the underlying distribution complexity. This intermediary enables transparent access to distributed storage while maintaining simplicity for end applications.
2Reliability
If one copy of application data is stored on one storage device by default, then storage management is simple, but data redundancy and reliability are insufficient
Solution Approach 1:
Multiple copies (replicas) of data objects are merged across different storage nodes. The namespace service automatically manages the distribution and replication of data objects to multiple nodes, providing redundancy without requiring manual intervention. This merging approach ensures data reliability while the automated namespace service maintains operational simplicity.
Solution Approach 2:
The storage system implements self-service through automatic replication and recovery mechanisms. When data is written to one node, the namespace service automatically replicates it to other nodes. If a node fails, the system automatically recovers data from replicas without user intervention. This self-service capability provides robust redundancy while maintaining ease of operation.
3Adaptability or versatility
If different operating systems store data in different incompatible formats, then each operating system can optimize its own storage, but cross-platform data accessibility is limited
Solution Approach 1:
The storage system implements a universal namespace service that works across all operating systems and platforms. The namespace service provides a consistent interface and data format that is platform-independent, allowing applications on any operating system to access stored data uniformly. This universality enables cross-platform accessibility while the service handles format conversion and compatibility internally.
Data Source
AI summary
A distributed, web-services based storage system. A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object. The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object. For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object. A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node.