PCPI Differencing for Metadata Search Database Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current file systems face inefficiencies in rapidly updating metadata databases, leading to inconsistent data and significant processing time, especially in large storage systems, due to the need for frequent file system crawls which impact performance and require direct access to all data containers.
Innovation Solution
The implementation of a system and method using persistent consistency point image (PCPI) differencing to quickly identify changes in metadata, allowing for accelerated updates to metadata search databases by generating and maintaining metadata search databases through a search agent that serially reads inode files and log files to maintain consistency with the file system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If file system crawls are performed frequently to update metadata databases, then database consistency is improved, but system performance deteriorates and processing time increases
Solution Approach 1:
The system performs preliminary actions by creating consistency point images at predetermined intervals before metadata updates are needed. These images capture the state of data containers at specific points in time, allowing subsequent differencing operations to quickly identify changes without requiring full crawls of the file system.
Solution Approach 2:
The invention extracts only the necessary information for metadata updating by comparing consistency point images to identify changed data containers. Instead of examining all data containers during each update cycle, the system extracts change information through image differencing, significantly reducing processing requirements.
2Loss of information
If file system crawls are performed to update metadata, then complete metadata information is obtained, but processing time and computational resources increase significantly
Solution Approach 1:
The system creates copies of file system state information in the form of consistency point images at predetermined intervals. These images serve as snapshots that can be stored and compared later, eliminating the need to perform time-consuming crawls of the actual file system during metadata update operations.
Solution Approach 2:
Consistency point images are generated in advance at predetermined intervals, capturing the state of data containers before updates occur. This preliminary capture allows subsequent differencing operations to quickly identify changes without requiring access to or examination of the actual data containers during the update process.
3Measurement precision
If direct access to all data containers is performed during metadata updates, then accurate metadata is obtained, but system performance and availability are impacted
Solution Approach 1:
The invention extracts metadata update information indirectly by comparing consistency point images rather than directly accessing data containers during updates. The differencing operation identifies changed containers through image comparison, obtaining accurate metadata information without requiring direct access to the actual data containers.
Solution Approach 2:
The system performs preliminary capture of file system state in consistency point images before updates occur. This allows subsequent metadata updates to be performed by comparing images rather than by directly accessing and examining data containers, maintaining accuracy while avoiding performance impacts.
4Reliability
If frequent metadata updates are performed to maintain consistency, then database reliability is improved, but the frequency and duration of crawls increase system overhead
Solution Approach 1:
The system maintains reliability by creating and comparing copies of file system state (consistency point images) rather than performing crawls of the actual file system. These image copies can be stored efficiently and compared using differencing algorithms, reducing the computational overhead associated with frequent updates.
Solution Approach 2:
Consistency point images are created at predetermined intervals rather than continuously or on every change event. This periodic creation of images, combined with differencing-based updates, maintains metadata consistency while reducing the frequency and complexity of update operations compared to traditional crawl-based approaches.
Data Source
AI summary
A system and method accelerates update of a metadata search database using PCPI differencing. After first populating the search database, a search agent generates a PCPI and utilizes a PCPI differencing technique to quickly identify changes between inode files of first and second PCPIs. The differences are noted as modified metadata and are written to a log file, which is later read by the search agent to update the search database.