Distributed Content Indexing via Unified Database
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional content indexing systems face scalability issues due to their single-device architecture, require reconfiguration when backup data formats change, and necessitate synchronization between metadata and content index databases, leading to increased computing costs and potential errors.
Innovation Solution
An improved content indexing system combines backup and content index databases into a single database, allowing for parallel processing and distributed content indexing across multiple servers and media agents, enabling efficient indexing of various file types regardless of backup data format, and avoiding synchronization operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional content indexing systems use separate backup metadata database and content index database, then data organization is achieved, but synchronization overhead and computing costs increase
Solution Approach 1:
The patent combines the backup metadata database and content index database into a single unified database. This eliminates the need for synchronization between two separate databases, reducing computing overhead and potential errors while maintaining proper data organization through unified management of backup metadata and content index information.
Solution Approach 2:
The unified database serves multiple functions: storing backup metadata, maintaining content indexes, and supporting both backup operations and content search operations. This multi-functional approach replaces the need for separate specialized databases, reducing system complexity while maintaining organizational reliability.
2Device complexity
If conventional content indexing systems run on single computing device, then system simplicity is maintained, but scalability is limited
Solution Approach 1:
The patent segments the content indexing system into multiple independent computing devices that can operate in parallel. Each device can handle portions of the indexing workload, allowing the system to scale horizontally by adding more devices. The segmentation enables distributed processing while maintaining system manageability through modular architecture.
Solution Approach 2:
The system transitions from a single-device vertical architecture to a multi-device horizontal architecture. This dimensional change allows scalability by adding more computing devices across the network, enabling the system to handle larger volumes of backup data and support more users without being constrained by single-device performance limits.
3Ease of manufacture
If conventional content indexing systems are configured for specific backup data format, then initial compatibility is achieved, but adaptability to format changes is reduced
Solution Approach 1:
The unified database architecture and distributed indexing system are designed to handle multiple backup data formats through a standardized interface layer. The system can ingest backup data in various formats, convert them to a common internal representation, and maintain content indexes accordingly, enabling format flexibility without sacrificing initial compatibility with specific backup formats.
Solution Approach 2:
The system uses parameter-based configuration to adapt to different backup data formats. By changing configuration parameters rather than reconfiguring the entire system architecture, the system can accommodate format changes efficiently. The unified database schema allows flexible storage of format-specific metadata while maintaining consistent content indexing capabilities across different formats.
Data Source
AI summary
An improved content indexing (CI) system is disclosed herein. For example, the improved CI system may include a distributed architecture of client computing devices, media agents, a single backup and CI database, and a pool of servers. After a file backup occurs, the backup and CI database may include file metadata indices and other information associated with backed up files. Servers in the pool of servers may, in parallel, query the backup and CI database for a list of files assigned to the respective server that have not been content indexed. The servers may then request a media agent to restore the assigned files from secondary storage and provide the restored files to the servers. The servers may then content index the received restored files. Once the content indexing is complete, the servers can send the content index information to the backup and CI database for storage.