Distributed I/O Coordination via Centralized Metadata Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, coordinating access to shared metadata across multiple nodes with direct I/O operations is challenging due to the lack of central coordination among filter drivers on different nodes, leading to issues with metadata access and data transformation processes.
Innovation Solution
Implementing a method that allows nodes to switch between direct I/O and redirected I/O access, where metadata operations are routed through a centralized node, enabling coordinated access and management of metadata, encryption, and data transformation processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If direct I/O access is used by multiple nodes simultaneously, then I/O performance and access speed are improved, but coordination of metadata access becomes difficult and data integrity may be compromised
Solution Approach 1:
The patent introduces a coordinator node as an intermediary that manages metadata access for all nodes performing direct I/O operations. The coordinator receives metadata access requests from filtering drivers on various nodes, processes them centrally, and returns results, thereby enabling fast direct I/O while maintaining reliable metadata coordination through a centralized mediation point.
2Adaptability or versatility
If filtering drivers operate independently on different nodes, then system autonomy and distributed operation are improved, but awareness of other filters' operations is lost
Solution Approach 1:
The patent implements a feedback mechanism where filtering drivers on distributed nodes register their operations with the coordinator node. The coordinator maintains awareness of active filters and their operations, providing feedback information to relevant nodes about what other filters are doing. This allows nodes to operate autonomously while maintaining situational awareness of the distributed system state.
3Productivity
If direct I/O operations are performed without owner node mediation, then I/O operation speed is improved, but encryption and data transformation coordination becomes problematic
Solution Approach 1:
The patent employs preliminary action by requiring filtering drivers to register their data transformation and encryption operations with the coordinator node before direct I/O operations commence. The coordinator pre-coordinates these operations with the owner node, establishing a plan for how encrypted or transformed data will be handled. This allows fast direct I/O while ensuring that data transformation coordination is arranged in advance.
Data Source
AI summary
Coordinating methods of I/O access to a shared data store. A method includes at a node, in a distributed system, performing one or more I/O operations on the shared data store using direct I/O access on a virtual data container. Direct I/O access includes performing I/O operations directly from the node to the shared data store including not having an owner node perform the I/O operation on the shared data store on the node's behalf. The owner node is a different node than the node doing the direct I/O operation. The owner node accesses the shared data store through a local data container. The method further includes determining that one or more subsequent I/O operations should be performed using redirected I/O access, where I/O operations are directed through a centralized node. The method further includes indicating to nodes in the distributed system to switch to redirected I/O access method.