Reference Count Update Log for Cluster File Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In cluster file systems, updating block reference counts for tracking shared data extents often results in performance penalties due to multiple disk I/O operations and contention issues, especially during operations like file snapshot creation and deduplication.

Innovation Solution

Implementing a Reference Count Update Log (RCUL) that records reference count updates transactionally with fixed-size records, allowing these updates to be performed asynchronously in the background, independent of the size of the data structure and number of nodes, thereby reducing I/O contention and enabling parallel updates across nodes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If b-tree updates are performed for tracking reference counts during file operations, then reference count tracking accuracy is improved, but system performance deteriorates due to multiple disk I/O operations

Engineering Contradiction:

Improvereference count tracking accuracyVSAvoidsystem performance

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent segments the reference count update process into two independent parts: (1) inline generation of reference count update log information during file operations, and (2) asynchronous application of these updates to the reference count table. This segmentation allows the accurate tracking function to be separated from the performance-critical path, eliminating the need for synchronous disk I/O during file operations while maintaining reference count accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary action by generating reference count update log information during inline file operations but deferring the actual application of these updates to a later asynchronous time. This allows the system to prepare all necessary update information without immediately performing the costly disk I/O operations, thereby improving performance while ensuring accuracy when updates are eventually applied.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If b-tree reference count updates are performed during cluster file system operations, then reference count accuracy is improved, but I/O contention increases

Engineering Contradiction:

Improvereference count accuracyVSAvoidI/O contention

Core Design Contradiction:

Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The patent introduces an intermediary component - the reference count update log - that acts as a buffer between file operations and the reference count table. This log captures all reference count update information during inline operations without immediately accessing the reference count table, thereby eliminating I/O contention during critical file operations while ensuring accurate updates are preserved for later application.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Stability of the object's composition

If reference count updates are performed inline during file operations, then data consistency is improved, but operation speed deteriorates

Engineering Contradiction:

Improvedata consistencyVSAvoidoperation speed

Core Design Contradiction:

Stability of the object's compositionVSSpeed

Solution Approach 1:

The patent performs preliminary action by generating reference count update log information during inline file operations to maintain data consistency, but defers the actual application of these updates to an asynchronous background process. This approach preserves data consistency by ensuring updates are captured and applied, while improving operation speed by eliminating the synchronous waiting time for disk I/O operations to complete.

Inventive Principle:

Principle #10Preliminary action

4Ease of operation

If b-tree structure is used for tracking reference counts, then reference count management is improved, but the structure becomes a source of contention in cluster environments

Engineering Contradiction:

Improvereference count managementVSAvoidcontention

Core Design Contradiction:

Ease of operationVSObject-affected harmful factors

Solution Approach 1:

The patent segments the reference count management system into two independent components: (1) the b-tree structure for tracking reference counts, and (2) the reference count update log for capturing updates. By separating the update capture function from the update application function, the b-tree structure is no longer a source of contention during file operations, as updates are logged asynchronously without requiring immediate access to the b-tree.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS8195619B2Extent reference count update system and method

Publication Date: 2012.06.05 COHESITY INC

AI summary

Systems and methods for extent reference count updates are presented. In one embodiment; a reference count update method includes: receiving an indication of new reference association with an extent of a shared storage component; generating reference count update log information for a reference count update log to indicate the new reference association with an extent of a shared storage component, wherein the altering occurs inline; forwarding a successful data update indicator to initiator of activity triggering the new reference association with the extent of the shared storage component; and updating a reference count table in accordance with the information in the reference count update log, wherein the updating is included in a background process.