Distributed Lock Manager Bitmap for Cluster Cache Coherency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In clustered environments, existing caching solutions are limited by the lack of interaction between disjoint caches, leading to increased latency and reduced throughput due to the inability to share data and access information, resulting in unnecessary I/O operations with slower backend storage devices.
Innovation Solution
The method involves determining and establishing an owner node within a cluster to manage data object ownership, using a distributed lock manager to ensure cache coherency, and selecting an owner node based on various policies such as random, circular, or cache characteristic methods, allowing for efficient data access and reduction of I/O operations by utilizing local caches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If local caches are implemented at each node, then processing load is distributed and single points of failure are minimized, but data sharing between nodes is prevented causing increased I/O latency
Solution Approach 1:
A distributed lock manager is introduced as an intermediary component that coordinates between disjoint caches in a cluster. The lock manager maintains a bitmap data structure to track which nodes hold locks on data objects, enabling coherent data sharing across nodes while preserving the reliability benefits of distributed caching. When a node needs data, it queries the lock manager to determine if another node has cached the data and can serve the request.
2Ease of operation
If disjoint caches in a cluster operate independently, then each node maintains its own cache locally, but data cannot be shared between nodes resulting in reduced throughput
Solution Approach 1:
The distributed lock manager implements a feedback mechanism where nodes query the lock manager's bitmap to determine data ownership status. When a node requests data, the lock manager provides feedback about which node currently holds the lock and can serve the data. This enables automatic data sharing decisions without complex manual coordination, improving throughput while maintaining operational simplicity.
Solution Approach 2:
The distributed lock manager serves multiple functions: it tracks data ownership across the cluster, mediates data sharing requests, maintains cache coherency, and enables efficient data routing. This multi-functionality allows disjoint caches to share data effectively without requiring separate mechanisms for each function, thereby improving throughput while keeping the system manageable.
3Stability of the object's composition
If no data sharing mechanism is implemented between caches, then cache coherence is maintained locally, but unnecessary I/O operations to backend storage occur increasing latency
Solution Approach 1:
The distributed lock manager acts as an intermediary that prevents unnecessary I/O operations by first determining whether data is already cached at another node before initiating a backend storage read. The lock manager's bitmap tracking mechanism enables efficient data sharing decisions, maintaining cache coherence while improving I/O efficiency by avoiding redundant reads from backend storage.
Data Source
AI summary
Disclosed herein are methods, systems, and processes to provide coherency across disjoint caches in clustered environments. It is determined whether a data object is owned by an owner node, where the owner node is one of multiple nodes of a cluster. If the owner node for the data object is identified by the determining, a request is sent to the owner node for the data object. However, if the owner node for the data object is not identified by the determining, selects a node in the cluster is selected as the owner node, and the request for the data object is sent to the owner node.