Delta Replication of Index Fragments for Disaster Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud environments face challenges in efficiently indexing and searching large volumes of structured data due to increased data quantity and size, which complicates disaster recovery processes.
Innovation Solution
The implementation of a unique sharding technique with delta replication of index fragments, allowing for re-sharding on a pre-defined schedule to enhance disaster recovery by ensuring quick data recovery and maintaining high availability, scalability, and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in dedicated databases with traditional indexing, then data storage and searching is efficient, but disaster recovery capability deteriorates due to lack of replication
Solution Approach 1:
The patent segments the database index into multiple index fragments that can be independently replicated. Each fragment represents a portion of the index data that can be copied to backup locations, enabling selective replication and faster recovery without replicating the entire database structure.
Solution Approach 2:
The system performs preliminary actions by continuously replicating index fragments to backup locations before disasters occur. This pre-positioning of replicated data enables rapid recovery operations when failures happen, reducing recovery time without requiring complex real-time replication systems.
2Quantity of substance
If data volume and size increase in cloud environments, then data storage capacity is improved, but indexing and searching efficiency deteriorates
Solution Approach 1:
The patent divides large datasets into smaller index fragments that can be processed and searched independently. This segmentation allows the system to handle large data volumes by processing smaller units, maintaining search efficiency even as total data quantity increases.
Solution Approach 2:
The system dynamically manages index fragments based on data volume changes. When data is added or modified, only the affected fragments need to be updated or recreated, rather than rebuilding the entire index. This dynamic approach maintains search efficiency as data volume grows.
3Reliability
If full database replication is implemented for disaster recovery, then recovery reliability is improved, but replication time and resource consumption increase
Solution Approach 1:
The patent extracts only the essential index fragments for replication rather than replicating the entire database. This extraction approach focuses replication resources on the most critical data structures needed for quick recovery, reducing replication time and resource consumption while maintaining recovery reliability.
Solution Approach 2:
The system implements partial replication by copying only the necessary index fragments rather than the complete database. This partial action is sufficient for disaster recovery purposes, achieving reliable recovery without the time and resource costs of full database replication.
4Adaptability or versatility
If static database structures are used, then system simplicity is maintained, but adaptability to cloud environments deteriorates
Solution Approach 1:
The patent introduces dynamic elements to the database structure by enabling flexible fragmentation and replication of index fragments. This dynamic structure allows the system to adapt to cloud environment requirements such as horizontal scaling, distributed storage, and flexible data access patterns while maintaining manageable complexity through modular fragment design.
Data Source
AI summary
In an example embodiment, a disaster is detected at a primary data center and, in response to the detection, a system switches over from the primary data center to a secondary data center such that searches from one or more client applications are routed to the secondary data center. Then, for each document stored in a search core of the secondary data center: a count is requested for the document from a first client application, it is determined whether the count for the document from the first client application matches a count for the document from the search core of the secondary data center, and, in response to a determination that the count for the document from the first client application does not match a count for the document from the search core of the secondary data center, a full publish for the document is requested from the first client application.