Cache Node Change Notification for Distributed File Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In distributed cache systems, cache clusters frequently perform unnecessary lookups for metadata from the home cluster, leading to latency even when data has not changed, as they periodically refresh metadata without notifications of changes.

Innovation Solution

Implementing a notification system where the source node updates a cache control file to indicate changes, allowing the cache node to request metadata or data only when an application requests it, thereby reducing unnecessary requests and latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the cache cluster frequently sends lookup requests to the home cluster to retrieve updated metadata, then the cache cluster maintains synchronization with the home cluster, but the latency increases due to unnecessary requests when data has not changed

Engineering Contradiction:

Improvesynchronization accuracyVSAvoidlookup latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The home cluster sends notifications to the cache cluster when metadata changes occur, creating a feedback mechanism that eliminates the need for frequent polling requests. The cache cluster receives change notifications and updates its cache accordingly, reducing unnecessary lookups while maintaining synchronization accuracy.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The home cluster proactively sends notifications to the cache cluster before the cache cluster would need to perform a lookup. This preliminary action ensures the cache cluster is already updated with the latest metadata when an application requests it, eliminating the need for latency-inducing lookup requests.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If the cache cluster uses a refresh interval to periodically lookup metadata from the home cluster, then the cache cluster maintains updated information, but unnecessary requests are sent when data has not changed

Engineering Contradiction:

Improvemetadata freshnessVSAvoidrequest efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

Instead of periodic polling, the home cluster provides feedback notifications only when metadata actually changes. This feedback mechanism ensures the cache cluster receives updates only when necessary, maintaining metadata freshness while eliminating wasteful requests that reduce system productivity.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The invention replaces periodic polling with event-driven periodic action. Rather than sending requests at fixed intervals regardless of change, the system sends notifications only when metadata changes occur, optimizing both freshness and efficiency.

Inventive Principle:

Principle #19Periodic action

3Reliability

If the cache cluster sends frequent Remote Procedure Calls to the home cluster to retrieve updated contents, then the cache cluster stays current with the home cluster, but the system experiences increased latency and overhead

Engineering Contradiction:

Improvedata currentnessVSAvoidsynchronization delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The home cluster implements a feedback mechanism that sends notifications to the cache cluster when data changes occur. This eliminates the need for frequent RPC calls while maintaining data currentness, as the cache cluster is notified and updated only when actual changes happen at the home cluster.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The cache cluster maintains its own metadata and only contacts the home cluster when notified of changes. This self-service approach reduces dependency on frequent RPC calls, maintaining data currentness through autonomous operation punctuated by change notifications.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11650957B2Receiving at a cache node notification of changes to files in a source file system served from a cache file system at the cache node

Publication Date: 2023.05.16 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Provided are a computer program product, system, and method receiving at a cache node notification of changes to files in a source file system served from a cache file system at the cache node. A cache file system is established at the cache node as a local share of a source file system at the source node. The source node establishes a local share of the cache file system at the cache node. Notification is received, from the source node, that the source node modified a source control file for a source file at the source node. In response to receiving the notification, a cache control file, for a cached file in the cache file system, is updated to indicate the source file at the source node is modified. A request is sent to the source node to obtain data for the source file indicated as modified.