Index Hint Generation for Legacy Backup Search Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing index processing methods for backup and restore operations in large storage systems are inefficient, leading to slow searches due to the need to examine the entire index, especially when dealing with hundreds of thousands or millions of objects, and are not effective for systems without file-like names or distinctive savepoints.
Innovation Solution
The implementation of search optimization techniques, such as concatenation, hashing, and bitmap indexing, to provide hints that facilitate faster lookup by summarizing and indexing backup item names, reducing the need to search the entire index and improving search efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a straightforward search of the index is performed to locate backup files, then the backup system can find backup objects, but the search becomes unwieldy and slow when dealing with hundreds of thousands or millions of objects
Solution Approach 1:
The patent segments the index into multiple partitions or subsets, allowing the search to be divided into smaller, more manageable segments. Instead of searching the entire index sequentially, the search is distributed across multiple partitions, reducing the time required to locate backup files in large storage systems.
Solution Approach 2:
The patent implements preliminary indexing and categorization of backup objects during the backup process itself. By pre-organizing data with metadata, tags, and hierarchical structures before retrieval operations, the system eliminates the need for slow sequential searches when restoration is needed, significantly improving locate speed.
2Productivity
If the index stores detailed information about all backup files to enable rapid searching, then lookup efficiency improves, but the index becomes unwieldy and complex to manage
Solution Approach 1:
The patent divides the index into multiple hierarchical levels or partitions, where each level stores a subset of information. This segmentation allows efficient lookup by narrowing down searches at each level while avoiding the need to store and manage a single monolithic complex index containing all details.
Solution Approach 2:
The patent introduces hierarchical dimensions to the index structure, organizing backup file information across multiple levels (e.g., volume level, file level, metadata level). This dimensional organization enables efficient lookups by navigating through hierarchical layers rather than searching a flat complex structure, improving productivity while managing complexity.
Data Source
AI summary
A method, article of manufacture, and apparatus for tracking a plurality of objects being stored are disclosed. In an embodiment, this comprises computing the hash value of the name of each object being stored, transforming the hash value into a plurality of bytes such that none of the bytes has the value of a flag used by the system, concatenating the transformed hashed values into a hint, and storing the hint in an index. In an embodiment, bytes having the flag value are mapped to an unused value during the transformation. In an embodiment, the hint is retrieved from the index and hashed values are transformed back. Mapped values are restored to the flag values. This allows use of the hint with a system that uses a flag in the index as an indicator; for example, to indicate that an endpoint has been reached.