Replicated Object Storage Transparent Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In replicated object storage systems, client applications face additional burdens when an object is not available on the primary system, requiring them to redirect to a replica, which can lead to increased workload and potential data loss due to node failures or corruption.
Innovation Solution
A replicated object storage system or content platform takes on the responsibility of retrieving and repairing damaged or unavailable objects from replicas, enabling a 'read from replica' feature that allows it to serve client requests from replicas when the primary system is unavailable, thereby reducing the client's workload and ensuring data availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the client application redirects to replica to access object when primary system is unavailable, then data availability is improved, but client application workload increases
Solution Approach 1:
The storage system automatically performs object recovery from replicas without client intervention. When an object is unavailable on the primary node, the system's recovery module autonomously retrieves the object from a replica node and restores it, eliminating the need for client applications to implement redirect logic or manage failure recovery manually.
Solution Approach 2:
The storage system acts as an intermediary between the client application and the replica nodes. Instead of the client directly redirecting to replicas, the storage system's recovery mechanism mediates the process by detecting unavailability, retrieving objects from replicas, and presenting them to clients transparently, thereby shielding clients from complexity while maintaining availability.
2Reliability
If the system automatically recovers objects from replicas, then service continuity is improved, but system complexity increases
Solution Approach 1:
The recovery functionality is merged with the existing storage system architecture. The recovery module integrates with the storage manager and node communication mechanisms already present in the system, combining object retrieval, validation, and restoration operations into a unified process that leverages existing infrastructure rather than adding separate complex subsystems.
Solution Approach 2:
The system maintains replica copies of objects across multiple nodes as a foundation for automatic recovery. These pre-existing copies serve as ready-to-use resources when primary objects become unavailable, eliminating the need for complex real-time data reconstruction or external recovery mechanisms, thereby simplifying the overall system architecture.
3Reliability
If the system reads objects from replicas during failures, then data accessibility is improved, but data consistency risks increase
Solution Approach 1:
The recovery mechanism incorporates feedback loops that monitor object availability and integrity. When objects are retrieved from replicas, the system validates their consistency and updates the state of replica nodes based on recovery outcomes. This feedback ensures that data accessed from replicas maintains consistency with the overall storage system state, preventing data corruption or inconsistency.
Data Source
AI summary
A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. An object recovery method comprises: receiving, by a first system of the plurality of systems from a client application, a read request for an object, the object having been replicated to/from at least one second system among the plurality of systems; if the object of the read request is available in the first system, returning by the first system the object of the read request to the client application; and if the object of the read request is not available in the first system, performing a read from replica process by the first system to access a replica of the object from a second system among the plurality of systems and using the replica of the object to return the object of the read request to the client application.