Change Log Compaction Using File Content Location Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file replication systems in distributed file systems face challenges in accurately identifying and managing files with hard links and renamed files, leading to redundant copying and incorrect file modifications due to reliance on file names rather than unique physical location identifiers.
Innovation Solution
The use of file content location identifiers, such as inode identifiers, to uniquely identify files and directories, allowing for efficient file replication, change log maintenance, and compacting, by referencing physical locations instead of file names, and creating file name aliases to manage hard links and renaming operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If file names are used to identify files in distributed file systems, then file operations can be performed using human-readable names, but redundant copying occurs and data integrity is compromised when files have hard links or are renamed
Solution Approach 1:
The patent introduces content location identifiers (such as inode identifiers) as an intermediary between file names and physical file locations. These identifiers serve as a mediator that remains stable across renaming and hard link operations, allowing the system to maintain reliable file identification while still using user-friendly file names for operations. The content location identifier acts as the intermediary that resolves the contradiction between ease of operation and reliability.
2Productivity
If file names are used for file replication, then replication can be performed based on user-visible names, but redundant copies are created when files are renamed or have multiple hard links
Solution Approach 1:
The patent implements a feedback mechanism where the system checks the content location identifier before performing file replication. When a file is renamed or new hard links are created, the content location identifier provides feedback that the physical location remains the same, allowing the replication system to skip redundant copying operations. This feedback loop enables the system to maintain high productivity while avoiding storage overhead from redundant copies.
Solution Approach 2:
The patent creates a mapping or alias between file names and content location identifiers, allowing the system to copy or reference files based on their stable identifiers rather than their names. This copying approach at the identifier level enables efficient replication while preventing redundant physical copies, as the content location identifier points to the actual physical storage location.
3Ease of operation
If change logs track file operations by name, then operations can be logged with user-friendly identifiers, but the change logs become difficult to manage and compact when files are renamed or linked
Solution Approach 1:
The patent changes the key parameter used in change logs from file name to content location identifier. This parameter change simplifies change log management because content location identifiers remain stable across renaming and hard link operations, eliminating the complexity of tracking name changes and link relationships. The change log can be compacted more effectively when using stable identifiers rather than volatile names.
Data Source
AI summary
Systems and methods for compacting change logs using file content location identifiers. An example method may comprise: identifying a first change log record comprising an identifier of a first data structure referencing a first physical location of contents of a file before a first file system operation, the first change log record further comprising an identifier of a second data structure referencing a second physical location of the contents of the file after the file first system operation; identifying a second change log record comprising the identifier of the second data structure referencing the second physical location of the contents of the file before a second file system operation, the second change log record further comprising an identifier of a third data structure referencing a third physical location of the contents of the file after the second file system operation; and replacing the first change log record and the second change log record with a third change log record comprising the identifier of the first data structure and the identifier of the third data structure.