Metadata Server Cluster Consistency via Deletion File List
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale storage systems with multiple metadata servers, asynchronous copying of metadata leads to inconsistencies, causing clients to potentially miss newly generated data or incorrectly assume data has been deleted, resulting in degraded performance and scalability issues.
Innovation Solution
A metadata server cluster with a master-slave structure, where a deletion file list unit stores information about deleted metadata for a predetermined period, and a metadata server selector chooses the appropriate server for operations based on this information to ensure consistent results, with the master server handling deletions and slaves handling checks, and re-routing requests if inconsistencies are detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a client accesses any metadata server randomly to increase scalability, then the system can handle more clients, but the client may not find desired metadata due to asynchronous copying delays
Solution Approach 1:
The system performs preliminary actions by having the master metadata server proactively push deleted metadata information to slave servers before a client might access them. This advance notification ensures that when clients access any slave server, the deletion information is already synchronized, preventing inconsistencies without requiring clients to check with the master server first.
Solution Approach 2:
The system implements feedback mechanisms where slave metadata servers report their synchronization status and metadata state back to the master server. This feedback loop allows the master server to track which slaves have received deletion information and to re-transmit information to any slaves that fall behind, ensuring eventual consistency across the cluster.
2Productivity
If metadata is asynchronously copied to all metadata servers, then system scalability is improved, but metadata disagreement occurs within a certain period
Solution Approach 1:
The master metadata server performs preliminary actions by immediately pushing deletion information to all slave servers as soon as a deletion occurs. This proactive approach ensures that deletion metadata is distributed before any client can access the slave servers, maintaining consistency while preserving the asynchronous architecture's scalability benefits.
Solution Approach 2:
The system provides beforehand cushioning by maintaining a buffer period where deleted metadata information is held and propagated to all slaves before being considered fully deleted. This cushioning period allows the system to handle any access requests that might occur during the brief synchronization window, ensuring no client encounters inconsistent state.
3Ease of operation
If a client accesses a metadata server during asynchronous copying, then the client may misjudge that deleted data exists or newly generated data does not exist
Solution Approach 1:
When a client accesses a slave metadata server, the server provides feedback about the current synchronization status. If the slave server detects that it has not yet received the latest deletion information from the master, it feeds back this status to the client or redirects the client to the master server, ensuring the client receives accurate metadata information regardless of which server it initially contacts.
Solution Approach 2:
The master metadata server acts as an intermediary between clients and slave servers for deletion operations. When a deletion occurs, the master server mediates by pushing the deletion information to all slaves and tracking their synchronization status. This intermediary role ensures that clients accessing any slave server receive accurate information about deleted data, as the master has already coordinated the deletion propagation.
Data Source
AI summary
Provided are an apparatus and a method for acquiring a consistent result even if one client accesses any metadata server in a circumstance that the cluster configuring many metadata servers exist. The apparatus as mentioned above includes a deletion file list unit that stores information about a deleted metadata during a predetermined period; and a metadata server selector that selects a metadata server to receive a operation request about a metadata among a plurality of metadata servers based on information from the deletion file list unit.