Adaptive Index Processing for Backup Search Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing index processing methods for backup and restore operations in large storage systems are inefficient, leading to slow searches due to the need to examine the entire index, especially when dealing with hundreds of thousands or millions of objects, and are not effective for systems without file-like names or distinctive savepoints.
Innovation Solution
The implementation of search optimization techniques, such as concatenation, hashing, and bitmap indexing, to provide hints that facilitate faster lookup by summarizing and indexing backup item names, reducing the need to search the entire index and improving search efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a straightforward search of the index is performed to locate backup files, then the backup system can find backup objects, but the search becomes unwieldy and slow when dealing with hundreds of thousands or millions of objects
Solution Approach 1:
The patent segments the index into multiple partitions or chunks, allowing the search to be distributed across multiple processors or threads simultaneously. Each partition contains a subset of the index entries, and search operations can be performed in parallel on different partitions, significantly reducing the overall search time for large numbers of objects.
Solution Approach 2:
The patent performs preliminary indexing and organization of backup object metadata before search operations are needed. The index is pre-processed and structured to enable faster retrieval, with metadata such as object names, paths, and timestamps organized in an optimized format that reduces the time required for search operations.
2Reliability
If the entire index is examined to locate backup objects, then all backup files can be found, but the search process becomes unwieldy and inefficient
Solution Approach 1:
The patent implements local quality by creating targeted search indexes for different types of backup objects or different time periods. Instead of examining the entire index uniformly, the search operation can focus on specific partitions or subsets of the index that are relevant to the search criteria, maintaining reliability by ensuring the correct subset is searched while improving efficiency through localized processing.
Solution Approach 2:
The patent applies partial action by searching only the necessary portions of the index rather than the entire index. Search operations can be limited to specific partitions, time ranges, or object types based on the search criteria, reducing the amount of work required while still finding all relevant backup objects. This may involve searching a subset of the index that is sufficient for the given query.
3Adaptability or versatility
If traditional indexing methods are used for systems with non-file-like names or numerous identical savepoints, then the index can be maintained, but search effectiveness is reduced
Solution Approach 1:
The patent changes the parameters used for indexing by incorporating multiple attributes beyond just object names, such as object paths, timestamps, file sizes, and other metadata. For systems with non-file-like names or identical savepoints, the index uses alternative or additional parameters to distinguish between objects, such as combining multiple attributes into a composite key or using hierarchical indexing that considers both name and path components.
Data Source
AI summary
A method, article of manufacture, and apparatus for tracking a plurality of objects being stored are disclosed. In an embodiment, this comprises determining an application associated with the objects being stored, selecting a hint or longest pathname containing the plurality of objects, and storing the selection in the index. The hint may be generated by concatenating at least a portion of a name of each object being stored into the hint, concatenating hash values of the names of the objects, or producing a bitmap of hash values of the names of the objects, depending on available space. In an embodiment, each hint generation may be tried in succession based on determining whether the previous hint fits into the space available for storing the hint.