Metadata Summary Entries for Faster Cluster Garbage Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Metadata scanning in data storage clusters consumes significant time and computing resources, acting as a bottleneck and slowing down data management actions, particularly in garbage collection processes.
Innovation Solution
Generating summary entries that include a subset of metadata information, allowing for faster data management actions by scanning these summaries instead of full metadata entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full metadata entries are scanned for data management decisions, then complete and accurate information is available for decision-making, but the scanning process consumes significant time and computing resources
Solution Approach 1:
The patent segments metadata into two distinct components: summary entries containing frequently accessed decision-making fields and full metadata entries containing complete information. This segmentation allows the system to scan only the smaller summary entries for routine data management decisions, significantly reducing scanning time while maintaining access to complete metadata when needed.
Solution Approach 2:
The patent extracts essential decision-making fields from full metadata entries to create separate summary entries. By taking out only the critical information needed for background decisions (such as space availability, data age, and access patterns), the system enables fast scanning without sacrificing the completeness of information required for accurate decision-making.
2Reliability
If full metadata entries are scanned for data management actions, then accurate decisions can be made, but computing resources are significantly consumed
Solution Approach 1:
The metadata is segmented into summary entries and full entries, allowing the system to use summary entries for routine decisions with lower computing resource consumption while maintaining reliability through selective use of full metadata when complex decisions require complete information.
Solution Approach 2:
Critical decision-making fields are extracted into summary entries, enabling accurate data management decisions to be made using only the extracted essential information, thereby significantly reducing computing resource consumption while maintaining decision accuracy for routine operations.
3Productivity
If summary entries with subset information are used for data management actions, then scanning time and resource consumption are reduced, but complete metadata information may be needed for certain decisions
Solution Approach 1:
The system segments metadata into summary and full entries, enabling fast scanning of summaries for routine decisions while preserving the option to access complete metadata when specific decisions require additional information, thus balancing productivity with information completeness.
Solution Approach 2:
Summary entries act as an intermediary layer between the scanning process and full metadata. This intermediary enables fast initial assessment and decision-making for routine operations, while providing a efficient pathway to access complete metadata when the intermediary summary information proves insufficient for complex decisions.
Data Source
AI summary
Techniques for pre-processing metadata for faster background decisions include, in response to a metadata entry associated with storage of data in a cluster being modified, generating a summary entry corresponding to the metadata entry. The summary entry includes a subset of the information from the metadata entry. A data management action is performed based upon a scanning process that reads a plurality of summary entries corresponding to a plurality of metadata entries for data in the cluster.